AxonHub is a production-grade, multi-provider LLM API gateway that routes, transforms, and manages requests to a wide range of AI providers (OpenAI, Anthropic, Gemini, DeepSeek, xAI, and 60+ others). It provides a unified API surface for chat completions, embeddings, moderations, audio, images, and video generation, with sophisticated features like load balancing (latency-aware, weighted round-robin, circuit breaker), per-channel rate limiting, provider quota enforcement, streaming, and reasoning token handling. The backend is written in Go using Gin, Ent ORM, gqlgen, and Uber Fx for dependency injection, supporting PostgreSQL, MySQL, and SQLite with automatic migration. The frontend is a comprehensive React/TypeScript SPA (Vite, TanStack Router/Query, Tailwind, shadcn/ui) that provides administration of channels, models, API keys, users, roles, projects, and system settings. It includes analytics dashboards, request tracing, usage logs, and OIDC SSO. The system is designed for multi-tenancy with fine-grained RBAC (scope-based permissions) and privacy policies enforced at the Ent ORM layer. Deployment is supported via Docker, Docker Compose, and Helm charts. The codebase exhibits strong engineering practices: extensive unit and integration tests (including race-condition tests and E2E Playwright tests), pipeline middleware for request transformation, generic caching abstractions, stream processing utilities, and a Zanzibar-inspired authorization framework. While there are minor issues (duplicate utility packages, a typo in a folder name, and some frontend boilerplate repetition), the overall architecture is clean, modular, and well-documented. The project is actively maintained with frequent commits, issue triage, and a clear release process.
LiteLLM is a production-grade, open-source proxy/gateway for over 200 large language model (LLM) providers, including OpenAI, Anthropic, Azure, Bedrock, Gemini, and many more. It presents a unified OpenAI-compatible API, handling request routing, authentication, rate limiting, cost tracking, caching, guardrails (content filtering, PII detection), and audit logging. The system is architected as a FastAPI Python server with a Next.js admin dashboard, Rust extensions for performance-critical paths (token counting, caching, auth, cost calculation), and optional PostgreSQL/Redis backends. It supports streaming, MCP (Model Context Protocol), A2A (Agent-to-Agent) protocol, and enterprise features like SSO, secret management, and Terraform deployment modules. Key entry points include the proxy server (litellm/proxy/proxy_server.py), CLI commands (litellm, litellm-proxy), and the admin UI at /ui/litellm-dashboard. The primary data flow is from client requests (chat completions, embeddings, audio, images) received by the proxy, which authenticates, applies guardrails, routes to the configured LLM provider(s), logs usage, and returns a standardized response. Caching is multi-tier (Redis, disk, S3, GCS, Azure Blob, semantic). Spend tracking is aggregated into daily tables and exposed via the management API and UI. External dependencies include PostgreSQL (via Prisma ORM), Redis (for distributed caching, rate limiting, and queuing), and cloud-specific secret managers (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault). The project uses uv for Python dependency management, Cargo for Rust crates, and npm/pnpm for the dashboard. Extensive CI/CD pipelines (CircleCI, GitHub Actions) run unit, integration, e2e, mutation, and Rust-Python parity tests.
Vigil is an open-source AI SOC (Security Operations Center) platform that orchestrates autonomous threat hunting, incident response, and case management through a multi-agent architecture. The system ingests security findings from SIEMs (Splunk, Elastic, Azure Sentinel), EDRs (CrowdStrike, Microsoft Defender), and other sources via Kafka or webhooks, then uses LLM-driven agents (hunt, root_cause, adjudicate, investigate, compose, chat) to analyze, enrich, and respond. A TypeScript agent layer with event-sourced ledgers and budget enforcement runs investigations, while a Python daemon handles polling, deduplication, and orchestration. The platform exposes its own tools via MCP (Model Context Protocol) and integrates with 30+ external security tools through a descriptor-based plugin system. The frontend is a React SPA with a setup wizard, dashboards, and case management UI. Deployment is via Docker Compose or Helm, with PostgreSQL, Redis, and optional Kafka as backing services.
This repository implements a production-grade semantic router for LLM inference, functioning as an Envoy ext_proc filter that intercepts chat completion requests, classifies them via multiple signals (intent, PII, jailbreak, domain, embeddings, etc.), and routes to the most appropriate model or model composition. The system supports Mixture-of-Models (MoM) with configurable algorithms (fusion, cascading, workflows) and provides advanced features like semantic caching, context compression, rate limiting, and observability. It includes a full-stack dashboard (React + Go backend) for configuration management, evaluation/experimentation (campaigns, A/B tests, capacity SLOs), and monitoring. A Python CLI and Kubernetes operator are available for deployment. The codebase demonstrates high engineering quality with extensive validation, testing, and a modular architecture, though its polyglot nature (Go, Rust, Python, TypeScript) and numerous optional backends introduce maintenance complexity. While research elements (training scripts, benchmarks) exist, the primary character is a deployable application.
Kong is a cloud-native, platform-agnostic API gateway built on OpenResty/Nginx. It provides traffic management, security, observability, and AI/LLM proxying capabilities through a plugin architecture. The system supports traditional, DB-less, and hybrid deployment modes, with an Admin API for configuration and Kong Manager for GUI-based management. Key features include load balancing, authentication, rate limiting, request/response transformation, and extensive observability integrations.
One API is a production-ready OpenAI API gateway that aggregates 30+ LLM providers (OpenAI, Anthropic, Google Gemini, Azure, Baidu, Alibaba, Zhipu, Moonshot, DeepSeek, Ollama, etc.) behind a single OpenAI-compatible endpoint. The Go backend implements a sophisticated relay system with provider-specific adaptors that translate OpenAI-format requests to each provider's native API. It features token-based quota management with group/channel multipliers, load balancing across channels, automatic failover, channel health monitoring with metric-based auto-disable, and multi-node deployment with Redis synchronization. The React frontend (three themes: default, berry, air) provides admin dashboards for channel/token/user management, billing visualization, and system configuration. The codebase demonstrates strong engineering practices: clean adaptor pattern for provider extensibility, comprehensive middleware chain, batched database updates for performance, and well-structured configuration management.
NetBird is a configuration-free peer-to-peer VPN and centralized access control system built on WireGuard. It consists of a cross-platform client daemon (Linux, Windows, macOS, Android, iOS, FreeBSD, WebAssembly), a management server, a signal server for NAT traversal coordination, a relay server for fallback connectivity, and a reverse proxy with an LLM gateway layer. The client daemon manages WireGuard interfaces, DNS, firewall rules, and peer connections via ICE/STUN/TURN, while the management server provides REST/gRPC APIs for account management, network map computation, posture checks, and policy enforcement. The proxy component adds middleware for authentication, LLM request parsing, cost metering, and guardrails, enabling identity-aware access to private resources and AI APIs over the encrypted tunnel. Architecturally, the system is a distributed set of microservices: the client runs as a background daemon with a CLI and a desktop GUI (Wails v3 + React), communicating with the management server via gRPC and REST. Relay and signal servers facilitate NAT traversal, and the proxy can be deployed as a standalone reverse proxy. The management server supports multiple database backends (PostgreSQL, SQLite, MySQL) and caching via Redis. Telemetry and metrics are exported to Prometheus or InfluxDB. The codebase is heavily platform-adapted with build-tagged files, and includes extensive unit, integration, and end-to-end tests. Key entry points include client/main.go (CLI), client/ui/main.go (desktop GUI), management/main.go (management server), relay/main.go (relay server), signal/main.go (signal server), proxy/cmd/proxy/main.go (proxy), and combined/main.go (all-in-one server). Data flows involve client authentication, network map retrieval, WireGuard tunnel setup, peer-to-peer connection establishment via ICE/relay, and optional proxy middleware processing for HTTP/LLM traffic. External dependencies include WireGuard, PostgreSQL/SQLite/MySQL, Redis, MaxMind GeoLite2, pyroscope, gRPC, protobuf, Wails, and various system tools (iptables, nftables, resolvconf).
This Rust codebase is an LLM inference proxy and tool-routing gateway. It exposes an OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) and forwards requests to multiple upstream LLM providers (OpenAI, Anthropic via direct/Bedrock, Vertex AI, vLLM DeepSeek, TokenHub) through a unified Provider trait. A QUIC server (s2n-quic, TLS 1.3) accepts connections from serverless tools written in Go or Node.js; the agent loop performs multi-round tool calling, invokes registered tools, and streams results back in SSE format. A separate model API proxy layer handles non-chat endpoint shapes (messages, responses, generateContent) with usage extraction and injection. Architecturally, the system is a Rust CLI binary (`yomo`) with three subcommands: `serve` starts the HTTP API and QUIC zipper, `init` scaffolds serverless tool projects, and `run` launches a tool subprocess and connects it to the zipper. Tool registration is done at runtime over QUIC and stored in-memory, so state is lost on restart. The HTTP layer is Axum, auth is bearer-token, and observability uses OpenTelemetry. The serverless execution model is subprocess-based: it creates temporary directories, runs `npm install`/`go mod tidy`, and exchanges metadata over stdout. Reusable engineering exists in the framing protocol (`io`), streaming SSE mappers, usage aggregation logic, TLS configuration, and provider error separation. However, the provider implementations are duplicated and brittle to upstream API drift; hard-coded model limits, ephemeral tool state, and near-empty serverless stubs reduce long-term maintainability. No integration tests against real upstream APIs were found in the reviewed surface.
CoAI (formerly ChatNio) is a full-stack AI chat platform and LLM gateway that provides a unified chat interface and OpenAI-compatible API proxy for 16+ AI model providers (OpenAI, Anthropic Claude, Google Gemini, Midjourney, DeepSeek, etc.). The system is built with a Go backend (Gin framework) and a React/TypeScript frontend (Redux, Tailwind CSS, Radix UI), with MySQL and Redis for persistence and caching. It supports real-time WebSocket chat, streaming responses, conversation management, file upload, web search, image generation, and a plugin system (MCP). The platform includes a comprehensive billing and subscription system with elastic per-token pricing and tiered plans (Basic/Standard/Pro), along with an admin dashboard for user management, channel configuration, analytics (model usage, request volume, revenue, errors), and system settings. It also features invitation codes, redeem codes, affiliate marketing, and multiple payment gateway integrations (Stripe, WeChat Pay, etc.). Architecturally, the backend employs an adapter pattern for AI providers, a channel manager for load balancing and retries, and a WebSocket connection manager with rate limiting. The frontend is heavily componentized with shadcn/ui and includes i18n support (English, Chinese, Japanese, Russian). Deployment is supported via Docker Compose, Zeabur, and a Tauri desktop app. While the codebase is well-structured and feature-rich, it shows signs of technical drift such as copy-pasted processor files across adapters, hardcoded model constants, and a large dependency surface. The open-source version is Apache 2.0 licensed, with a commercial 'CoAI Pro' tier offering additional features.
Magic is a large, polyglot enterprise AI-agent and collaboration platform. The production core is a PHP Hyperf backend (`backend/magic-service`) with strict DDD layering, a Python agent runtime (`backend/super-magic`) for tools/skills/browser automation, a Go engine for knowledge processing/embedding, and a React/TypeScript frontend (`frontend/magic-web`) that spans desktop, mobile, admin, and a visual flow editor. The platform delivers chat/messaging, AI agents, workflow automation (MagicFlow), knowledge bases with vector search, an LLM/model gateway, file storage, low-code data tables (MagicBase), MCP integration, and multi-channel bot deployment (DingTalk, Feishu, WeCom). It is fundamentally an application rather than a research repository; the knowledge-base and data-processing subsystems serve product features, not analysis. Architecturally, the strongest parts are the PHP library packages — async-event, cloudfile, flow-expr-engine, code-executor, and easy-dingtalk — which are cleanly separated and demonstrate genuine, extractable craftsmanship. The frontend likewise includes reusable packages for flow editing, HTML-to-PPTX conversion, and UI components. However, the repository is physically enormous and shows maintenance risk: substantial code duplication exists between `backend/magic-service` and `backend/super-magic-module`, several enterprise features are intentionally stubbed out with open-source no-ops, migration rollback methods are frequently empty, and the mix of PHP, Python, Go, and TypeScript adds integration complexity. The codebase is actively developed, but quality is uneven across the many modules.
ThinkWatch is a Rust workspace implementing an AI gateway platform that fronts multiple model providers and MCP (Model Context Protocol) servers. It exposes OpenAI-compatible chat/completion endpoints, streaming, MCP tool-call routing, content filtering, and provider failover. The backend is split into crates: `common` (shared lifecycle/limits/access), `auth` (PoW login, API keys), `gateway` (model routing), `mcp-gateway` (MCP server proxying), `server` (REST admin/audit/trace endpoints), and `test-support` (integration test client). A React/TypeScript frontend in `web/` provides an admin console, setup wizard, limits panels, and MCP server management. Data flows: requests enter the gateway, pass through `check_limits`, `check_budget`, `check_access`, then `run_post_invoke`; `trace_id` is propagated across gateway, MCP, audit, and app logs stored in ClickHouse; Postgres holds users, API keys, budgets, and MCP server configs; Redis/Lua scripts enforce rate limits.
ClawManager is a full-stack, Kubernetes-native control plane for managing AI agent runtimes (OpenClaw, Hermes, OpenCode, DeepSeek Harness, Workbuddy). It provides instance lifecycle management, secure AI gateway proxying with governance and cost tracking, skill management (import, versioning, hub), team collaboration with leader-mediated orchestration, and a comprehensive security protection platform (SecPlane) covering runtime behavior, egress policy, intrusion detection, and compliance reporting. The backend is built in Go with Gin, uses MySQL for persistence, Redis for real-time events, and the Kubernetes client-go library for orchestration. The frontend is a React 19 SPA with TypeScript, Vite, Tailwind CSS, and i18n support for five languages. The repository also includes Playwright-based E2E tests, Docker Compose local development stacks, and a Hermes desktop web component with its own build pipeline. While the codebase demonstrates solid architecture (repository pattern, service layer, middleware), some areas show drift (many SQL migrations, hardcoded dev secrets, incomplete feature flags) and the sheer number of runtime types and configurations introduces complexity.