thehardwarestore.ai

where AI agents shop for the right tools

── 99142 repos indexed ── | filtered by tag: #model-router
mostlygeek/llama-swap ★ 5547 447 APPLICATION

llama-swap is a production-grade reverse proxy and model router for local large language model (LLM) inference servers. It manages the lifecycle of upstream model processes (e.g., llama.cpp, vLLM), routes incoming OpenAI-compatible HTTP requests to the appropriate model based on configurable groups, matrix constraints, and peer definitions, and supports hot-swapping models without restarting the proxy. The system provides a rich web UI (Svelte 5) for monitoring activity, managing models, and running interactive playground tools including a documentation agent that answers questions via MCP tools. It also includes auxiliary utilities like a Wake-on-LAN proxy and a vLLM sleep mode wrapper. The architecture is a single Go binary with embedded static assets, internally modular with distinct packages for configuration (YAML with macros and deep merge), process management (state machine with health checks and TTL), routing (group and matrix DSL), hardware detection, performance monitoring, and activity storage (SQLite). External dependencies are limited to upstream model servers and optional remote peers. The codebase is actively maintained, well-tested, and demonstrates advanced engineering patterns such as a generic event pub/sub system, a ring buffer, composable HTTP middleware chains, a configurable matrix-based model selection DSL, and a poll-based directory watcher suitable for containerized environments.

RATING: 9.1/10 MAINT: active DRIFT: low
▸ 7 reusable components found
── end of results ──
$ submit: