$ show stacklok/toolhive
ToolHive is a production-grade MCP (Model Context Protocol) server management platform developed by Stacklok. It provides a comprehensive CLI (`thv`), a Kubernetes operator (`thv-operator`), a proxy runner, and a virtual MCP server (vMCP) for aggregating multiple backend MCP servers. The system manages the full lifecycle of MCP workloads—running, stopping, building, exporting, and monitoring—while enforcing authentication (OAuth, OIDC, SPIFFE, AWS STS), authorization (Cedar, HTTP PDP, annotation-based), rate limiting, telemetry (OpenTelemetry/Prometheus), and secrets management (1Password, keychain, environment). It supports multiple transport protocols (SSE, streamable HTTP, stdio) and container runtimes (Docker, Podman, Kubernetes). The architecture is modular: a CLI layer built with Cobra, an API layer (RESTful registry and health endpoints), a Kubernetes operator layer with CRD reconcilers, a transport proxy layer with session management, and a plugin/skill system for installing and verifying OCI artifacts with Sigstore signing. The vMCP subsystem provides tool aggregation, conflict resolution, capability caching, and health monitoring with circuit breakers. External dependencies include Kubernetes client-go, Docker engine, Redis/Valkey, OpenTelemetry SDK, Sigstore, Ory/Fosite (OAuth), and various Go libraries for YAML, JSON Schema, and HTTP networking. Key data flows: CLI commands trigger workload creation via the runner, which builds a RunConfig from user flags and environment, then deploys the MCP server in a container (Docker/Kubernetes). The proxy layer intercepts MCP JSON-RPC messages, applies middleware (auth, rate limiting, telemetry, body limits), and forwards to the backend server. The vMCP server aggregates tools from multiple backends, resolves naming conflicts, and presents a unified MCP interface to clients. Telemetry spans are created at each layer and exported via OTLP. Skills and plugins are installed from OCI registries, verified with Sigstore, and managed via lock files with content digest validation.
The LLM identified 5 files worth highlighting for reuse:
Provides a generic bounded concurrent iteration helper that limits the number of in-flight goroutines, with context cancellation support. The `ForEach` function accepts a collection, a concurrency limit, and a callback, then spawns a controlled number of goroutines to process items. It is self-contained, imports no internal modules, and can be extracted into any Go project needing safe, bounded parallelism without external dependencies.
Implements safe atomic file writes via a write-to-temp-then-rename strategy, ensuring data integrity even on crash or partial write. The `AtomicWriteFile` function writes content to a temporary file in the same directory, then renames atomically. It handles permission preservation and cleanup. This is a standalone utility reusable across any application that writes configuration, state, or data files.
Detects ambiguous JSON constructs (duplicate keys, case variants, Unicode folding) that could cause parser differentials between MCP client and server. The `CheckJSONAmbiguity` function scans a JSON byte stream for these patterns and returns an error if found. It demonstrates deep understanding of JSON parsing security and is reusable in any system that needs to prevent parser confusion attacks.
Implements a Redis-backed token bucket rate limiter using atomic Lua scripts for thread-safe consumption and refill. The `Bucket` struct provides `Allow` and `Reset` methods that execute Lua on the Redis server, ensuring atomicity. It supports configurable capacity, refill rate, and per-key isolation. This is a self-contained component reusable for any rate-limiting need, requiring only a Redis connection.
Generates JSON Schema from Go structs via reflection, used for MCP tool input validation. The `Reflect` function uses Go's `reflect` library to introspect struct fields, types, and documentation comments, producing a JSON Schema object. It handles nested structs, arrays, enums, and optional fields. This is a standalone utility that can be extracted into any Go project needing automatic JSON Schema generation from native types.