This repository implements a production-grade semantic router for LLM inference, functioning as an Envoy ext_proc filter that intercepts chat completion requests, classifies them via multiple signals (intent, PII, jailbreak, domain, embeddings, etc.), and routes to the most appropriate model or model composition. The system supports Mixture-of-Models (MoM) with configurable algorithms (fusion, cascading, workflows) and provides advanced features like semantic caching, context compression, rate limiting, and observability. It includes a full-stack dashboard (React + Go backend) for configuration management, evaluation/experimentation (campaigns, A/B tests, capacity SLOs), and monitoring. A Python CLI and Kubernetes operator are available for deployment. The codebase demonstrates high engineering quality with extensive validation, testing, and a modular architecture, though its polyglot nature (Go, Rust, Python, TypeScript) and numerous optional backends introduce maintenance complexity. While research elements (training scripts, benchmarks) exist, the primary character is a deployable application.
DeepTide is a cross-platform terminal AI coding agent. The Rust workspace (deeptide-core, deeptide-host, deeptide-cli, deeptide-gui) implements the production agent engine: an interactive REPL session with an agent loop, tool dispatch (bash, edit, web, git, memory, MCP, LSP, vision, cron, planning), cross-provider model routing with local-first/cloud-escalation policies, cost tracking, tokens-per-second telemetry, tool usage observability, durable memory with BM25 retrieval ranking, session import/checkpoint/rewind, sensitive-file read protection, and a theme system for syntax highlighting and diffs. The CLI and GUI share a common host layer, and the npm package downloads prebuilt Rust binaries. Architecturally it is a multi-crate Rust workspace plus supporting native components: native/ds4 is a hard-forked DeepSeek V4 Flash Metal inference engine for local LLM execution, and native/dsgo is a Swift gateway that translates between Anthropic and OpenAI protocols with SSE streaming. Primary entry points are crates/deeptide-cli/src/main.rs (CLI), crates/deeptide-gui/src/main.rs (eframe/egui GUI), and bin.js (Bun launcher that delegates to @paean-ai/zero-cli). External dependencies include DeepSeek/Anthropic/OpenAI-compatible APIs, local Metal GPU inference, and GitHub Releases for binary distribution. The repo also contains 50+ self-contained HTML5 Canvas pixel-art games and Three.js showcases under samples/, which appear to be AI-generated demo content demonstrating agent output quality rather than core product code. These share heavily duplicated boilerplate (seeded RNG, localStorage save/load, bilingual i18n, screen state machines). The core Rust code is well-tested with integration tests covering routing, cost tracking, memory ranking, tool result summarization, and slash commands; live API tests are marked #[ignore]. Hybrid note: while primarily an application, it contains research-oriented elements such as memory capture benchmarks (benchmarks/auto_capture_bench.py), prompt cache layout benchmarks (benchmarks/cache_memory_bench.py), and deterministic puzzle generation algorithms with uniqueness verification.
AIClient2API (A2) is a production-grade API proxy/gateway that converts proprietary AI provider APIs (Gemini, Claude/Kiro, Qwen, Codex, Grok, OpenAI, and more) into standard OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/messages`, `/v1beta/models`). It supports multi-account OAuth credential pools with automatic token refresh, health checks, and fallback routing, enabling clients to break free-tier restrictions. The server is built on Node.js with a master-worker architecture for process management, a plugin system for extensibility (authentication, monitoring, usage statistics, API key sharing), and a Go-based TLS sidecar for JA3/JA4 bypass. A full web management console (static HTML/CSS/JS) provides configuration, provider management, credential file upload, interactive playground, and usage monitoring. The codebase demonstrates strong patterns (strategy, factory, adapter, converter) but suffers from some monolithic utility files, hardcoded provider mappings, and high maintenance overhead from multiple OAuth implementations.
ThinkWatch is a Rust workspace implementing an AI gateway platform that fronts multiple model providers and MCP (Model Context Protocol) servers. It exposes OpenAI-compatible chat/completion endpoints, streaming, MCP tool-call routing, content filtering, and provider failover. The backend is split into crates: `common` (shared lifecycle/limits/access), `auth` (PoW login, API keys), `gateway` (model routing), `mcp-gateway` (MCP server proxying), `server` (REST admin/audit/trace endpoints), and `test-support` (integration test client). A React/TypeScript frontend in `web/` provides an admin console, setup wizard, limits panels, and MCP server management. Data flows: requests enter the gateway, pass through `check_limits`, `check_budget`, `check_access`, then `run_post_invoke`; `trace_id` is propagated across gateway, MCP, audit, and app logs stored in ClickHouse; Postgres holds users, API keys, budgets, and MCP server configs; Redis/Lua scripts enforce rate limits.
AgenticSeek is a private, local autonomous AI assistant built as an alternative to Manus AI. The system features a multi-agent architecture with six specialized agents (Casual, Coder, File, Browser, Planner, MCP) selected by a hybrid router using BART zero-shot classification and an adaptive few-shot classifier. It provides a React frontend, FastAPI backend with Celery, CLI with voice support, and Docker Compose orchestration for self-hosted services including SearxNG search and Redis caching. The codebase demonstrates solid engineering with clean abstractions for LLM providers (14+ options), tool execution, browser automation, and workspace isolation. It is under active development but shows signs of technical drift in deprecated components (llm_server, SerpApi-based web search) and fragile browser automation. The architecture is modular and contains several reusable patterns, though the codebase is tightly coupled to its specific agent system and has substantial external dependencies.