Headroom is a context optimization layer for LLM applications that reduces token usage by 50-90% through intelligent compression, caching, and memory systems. It provides a transparent proxy between LLM clients (like Claude Code, Cursor, etc.) and LLM providers (Anthropic, OpenAI, etc.), implementing CCR (Compress-Cache-Retrieve) for lossless compression with retrieval capability. The system features a hybrid polyglot architecture with a Rust-based high-performance proxy core, Python ecosystem for extensibility, and TypeScript SDK. Key capabilities include agent wrapping, cross-agent memory using Qdrant/Neo4j, observability through metrics and dashboard, and a learning system that analyzes interaction patterns to generate optimization rules. Deployment options include Docker, system services, and direct execution.
llama-swap is a production-grade reverse proxy and model router for local large language model (LLM) inference servers. It manages the lifecycle of upstream model processes (e.g., llama.cpp, vLLM), routes incoming OpenAI-compatible HTTP requests to the appropriate model based on configurable groups, matrix constraints, and peer definitions, and supports hot-swapping models without restarting the proxy. The system provides a rich web UI (Svelte 5) for monitoring activity, managing models, and running interactive playground tools including a documentation agent that answers questions via MCP tools. It also includes auxiliary utilities like a Wake-on-LAN proxy and a vLLM sleep mode wrapper. The architecture is a single Go binary with embedded static assets, internally modular with distinct packages for configuration (YAML with macros and deep merge), process management (state machine with health checks and TTL), routing (group and matrix DSL), hardware detection, performance monitoring, and activity storage (SQLite). External dependencies are limited to upstream model servers and optional remote peers. The codebase is actively maintained, well-tested, and demonstrates advanced engineering patterns such as a generic event pub/sub system, a ring buffer, composable HTTP middleware chains, a configurable matrix-based model selection DSL, and a poll-based directory watcher suitable for containerized environments.
The G0DM0D3 repository provides an AI chat interface for interacting with multiple LLMs via OpenRouter, Venice, or local models, featuring advanced capabilities like multi-model racing (Ultraplinian), hive-mind synthesis (Consortium), context-adaptive parameter tuning (AutoTune), input obfuscation (Parseltongue), and output transformation (STM). It includes a privacy-first telemetry system and opt-in dataset contribution for research. The architecture combines a standalone HTML option, a React/Next.js frontend, and a Node/Express backend with OpenAI-compatible API endpoints, middleware for authentication, rate limiting, and tier-based access control. Key reusable components include the telemetry beacon, feedback learning system, modular transformation engines, and tier-based access control pattern.
ClawRouter is a production-grade AI/LLM proxy gateway deployed on Cloudflare Workers. It provides a unified API endpoint for multiple AI providers (OpenAI, Anthropic, Google Gemini, etc.) with policy-based access control, per-user and per-provider budget accounting, usage tracking, and a comprehensive admin console (React/TypeScript). The system authenticates users via Cloudflare Access, local auth, or API keys, then routes requests through configurable policies that enforce budget limits, grant selection, and content retention. It supports streaming, WebSocket-based Responses API, and a Fusion mixture-of-agents feature for multi-model orchestration. The worker uses Durable Objects for stateful budget ledgers, usage ledgers, credential stores, and background job recovery, all backed by SQLite. A rich admin UI (Patchboard) allows operators to manage policies, credentials, grants, users, and view analytics. The codebase includes extensive unit, integration, and browser (Playwright) tests, along with CI/CD pipelines for Cloudflare deployment and self-hosting via Docker/workerd. It is a well-architected, thoroughly tested system that demonstrates high craftsmanship in state management, error handling, and idempotent accounting patterns.
EvoMap Evolver is a self-evolving AI agent engine built as a Node.js >=22.12 CLI daemon. It runs a genome-evolution loop: scans session memory/logs, extracts signals, selects 'Genes' (prompt/strategy patches) and 'Capsules' (validated skill bundles) from a local JSON/JSONL asset store, generates GEP-framed prompts for coding agents (Claude Code, Codex, Cursor, Gemini CLI, opencode, Kiro), executes those sessions, and records outcomes in a memory graph to adapt future selections. The system solidifies successful patterns into new reusable assets, auto-distills failure patterns and SKILL.md files into Genes/Capsules, and can open PRs or run validation sandboxes against candidate changes. It also ships an LLM proxy (Anthropic/OpenAI/Gemini/Vertex/Ollama routes with trace encryption and token-savings attribution), an ATP (Agent Transaction Protocol) marketplace for buying/completing agent tasks with auto-buyer and auto-deliver, a validation daemon for decentralized task verification, a local Web UI dashboard, and an experiment runner for A/B comparing evolution strategies. Hub synchronization (SSE event delivery, mailbox sync, OAuth device login) connects nodes to the evomap.ai network, but the system is designed to degrade gracefully into fully offline solo mode.
BankOfAI Pool is a full-stack application that automates the registration and management of API keys for the bankofai.io service. It reverse-engineers the authentication flow to create new accounts, claims the 500,000 credit bonus per account, and pools the resulting API keys behind a unified OpenAI-compatible proxy endpoint. The system includes a Next.js dashboard for managing the pool, automatic replenishment mechanisms, proxy rotation services to bypass IP-based rate limits, and intelligent quota management to distribute keys efficiently. The architecture separates concerns into frontend UI, backend API services, database storage, and specialized workers for registration and maintenance.
This repository implements an OpenAI-compatible REST API proxy for Amazon Bedrock, enabling applications built for OpenAI SDKs to seamlessly use Bedrock foundation models without code changes. The FastAPI-based service translates standard OpenAI API calls (chat completions, embeddings, models listing) into Amazon Bedrock runtime invocations and returns responses in OpenAI-compatible format. The architecture supports two deployment options: Lambda with API Gateway (serverless, cost-effective) or Fargate with ALB (lowest latency, no cold starts). Key features include streaming responses via SSE, tool/function calling, multimodal inputs, prompt caching for Claude/Nova models, reasoning mode for Claude 3.7 and DeepSeek R1, and interleaved thinking for Claude 4 models. Infrastructure is managed via CloudFormation templates with proper IAM roles and Secrets Manager integration for API key authentication. The codebase demonstrates solid engineering practices with clean separation of concerns, proper error handling, and comprehensive feature support. However, the absence of test files and limited documentation reduce overall quality. The project is actively maintained with recent commits and up-to-date dependencies.
ThinkWatch is a Rust workspace implementing an AI gateway platform that fronts multiple model providers and MCP (Model Context Protocol) servers. It exposes OpenAI-compatible chat/completion endpoints, streaming, MCP tool-call routing, content filtering, and provider failover. The backend is split into crates: `common` (shared lifecycle/limits/access), `auth` (PoW login, API keys), `gateway` (model routing), `mcp-gateway` (MCP server proxying), `server` (REST admin/audit/trace endpoints), and `test-support` (integration test client). A React/TypeScript frontend in `web/` provides an admin console, setup wizard, limits panels, and MCP server management. Data flows: requests enter the gateway, pass through `check_limits`, `check_budget`, `check_access`, then `run_post_invoke`; `trace_id` is propagated across gateway, MCP, audit, and app logs stored in ClickHouse; Postgres holds users, API keys, budgets, and MCP server configs; Redis/Lua scripts enforce rate limits.
This repository contains demonstration scripts and documentation for the GPT_API_free service, a forwarding proxy that provides free access to various large language models including GPT, DeepSeek, Claude, Gemini, and others. The actual forwarding server is not included; instead, the repository offers Python examples using the OpenAI library and standard HTTP requests to illustrate how to interact with the service's API endpoints for chat completions, image generation/editing, audio transcription, speech synthesis, and file analysis. The code is minimal and educational, focusing on replacing the base URL and authentication header to route requests through the proxy. The codebase is primarily a collection of usage examples tightly coupled to the external service's address and API key format. There is no production logic, no tests, and no independent reusable components. The demos follow standard patterns for calling OpenAI-compatible APIs and serve as a quick-start guide for developers integrating the service. As such, the repository does not represent a standalone application but rather a documentation supplement for the GPT_API_free project.
APIMyLlama is a Node.js application that serves as a proxy and management layer for the Ollama API. It provides an endpoint (/generate) to interact with LLMs using API keys, which are managed via a command-line interface. The application includes features such as API key generation, validation, rate limiting, webhook notifications, and dynamic configuration of server port and Ollama URL. The architecture is monolithic, leveraging Express for HTTP handling, SQLite for persistence, and a custom CLI for administrative tasks. Key components include rate limiting middleware and a webhook notification system, which demonstrate common patterns in API management. However, the project suffers from dependency issues where the version constraints for axios and express are set to versions (>=1.13.5 and >=5.2.1) that are not currently available, preventing installation with the latest stable packages. This indicates a lack of maintenance in terms of dependency updates, despite recent commits.
This repository contains only documentation and image assets for a tutorial on using CLIProxyAPI, a third-party tool for proxying large language model APIs. It does not include any source code, executables, or runnable components. The tutorial provides step-by-step instructions for downloading the CLIProxyAPI executable from an external release page, configuring it via a YAML file, and accessing its web interface to integrate with LLM services like Qwen. As a codebase, it lacks any actual implementation, making it purely a reference guide with no reusable code or architectural elements.