$ show songquanpeng/one-api
One API is a production-ready OpenAI API gateway that aggregates 30+ LLM providers (OpenAI, Anthropic, Google Gemini, Azure, Baidu, Alibaba, Zhipu, Moonshot, DeepSeek, Ollama, etc.) behind a single OpenAI-compatible endpoint. The Go backend implements a sophisticated relay system with provider-specific adaptors that translate OpenAI-format requests to each provider's native API. It features token-based quota management with group/channel multipliers, load balancing across channels, automatic failover, channel health monitoring with metric-based auto-disable, and multi-node deployment with Redis synchronization. The React frontend (three themes: default, berry, air) provides admin dashboards for channel/token/user management, billing visualization, and system configuration. The codebase demonstrates strong engineering practices: clean adaptor pattern for provider extensibility, comprehensive middleware chain, batched database updates for performance, and well-structured configuration management.
The LLM identified 8 files worth highlighting for reuse:
Clean adaptor interface defining the contract for all 30+ LLM provider integrations. Each provider implements Init, GetRequestURL, SetupRequestHeader, ConvertRequest, ConvertImageRequest, DoRequest, DoResponse, GetModelList, GetChannelName. This pattern enables adding new providers without modifying core relay logic - just implement the interface and register in relay/adaptor.go.
Shared OpenAI-compatible request utilities including ResponseText2Usage for token counting from response text, and GetFullRequestURL which handles complex URL routing for different channel types (OpenAI, Azure, Cloudflare AI Gateway). Demonstrates careful handling of provider-specific URL path variations.
Thread-safe in-memory rate limiter using sliding window algorithm with automatic expiration cleanup. Supports configurable max requests per time window per key. Used for API/web rate limiting middleware. The goroutine-based cleanup prevents memory leaks.
Batched database update system that aggregates quota consumption in memory and flushes periodically (configurable interval). Uses separate maps and mutexes per update type (user quota, token quota, used quota, channel quota, request count). Significantly reduces database write load under high throughput.
Channel health monitoring using sliding window success rate tracking. Configurable queue size and threshold. Runs background consumers for success/failure events. When success rate drops below threshold over the window, automatically disables channel and notifies admins via email/Message Pusher. Includes auto-re-enable on successful requests.
Redis client initialization supporting standalone, cluster, and sentinel modes. Parses connection string, handles authentication, master name for sentinel. Provides helper functions (Set, Get, Del, Decrease) with context. Gracefully degrades when Redis not configured.
Comprehensive authentication middleware validating JWT tokens, checking token status/expiry/quota/IP allowlist/model allowlist, supporting channel selection via token suffix (TOKEN-CHANNEL_ID), enforcing group/channel model permissions, and setting request context for downstream relay.
Thread-safe group-based billing ratio management with JSON serialization for config persistence. Supports runtime updates via API. Used to apply different pricing multipliers per user group (default, vip, svip) which combine with model ratios for final quota calculation.