$ show labring/aiproxy
aiproxy is a production-grade AI API gateway written in Go, acting as a unified proxy between client applications and over 40 AI provider APIs (OpenAI, Anthropic, Gemini, etc.). It converts protocol formats (OpenAI, Anthropic, Gemini), handles authentication, rate limiting, load balancing, request caching, and detailed usage billing. The system includes a React admin dashboard for managing channels, tokens, models, groups, logs, and consumption. It also supports the Model Context Protocol (MCP) for integrating AI tool servers, both hosted and proxied. The architecture is monolithic with a plugin system for extensibility, storing state in PostgreSQL and Redis. It demonstrates strong engineering practices including comprehensive test coverage, efficient caching (multi-layer with keyed locks), sliding window rate limiting, and a clean adaptor pattern for provider abstraction.
The LLM identified 2 files worth highlighting for reuse:
Implements a per-key mutual exclusion mechanism using sync.Map and channels. Provides Lock/Unlock and DoWithLock methods that allow concurrent access to shared resources keyed by string. The implementation is efficient and avoids global locks, making it easy to extract and reuse in any Go project needing fine-grained locking without external dependencies.
An in-memory sliding window rate limiter using atomic counters and a circular buffer. Tracks requests per second with configurable window size and limit. Uses only standard library packages (sync, atomic, time), making it highly reusable. The design is clean, thread-safe, and can be easily extracted as a standalone rate limiting library.