LocalAI is a modular AI inference server providing OpenAI-compatible APIs for running diverse models (LLMs, vision, audio, video) locally. It features a polyglot architecture with Go core services, pluggable backends (C++/Go/Python), and a React-based web UI. The system supports distributed deployment via NATS/PostgreSQL, model gallery management, and multimodal processing. Key capabilities include model loading from Hugging Face, quantization, PII detection, and real-time transcription. The codebase demonstrates strong engineering practices with dependency injection, middleware chains, and clean separation of concerns.
New API is a full-stack AI gateway/proxy service that provides a unified interface for multiple AI model providers (OpenAI, Claude, Gemini, etc.). It features user authentication and authorization, API key/token management with quota tracking and billing, administrative dashboard for configuration and monitoring, intelligent routing between providers, format conversion between different AI APIs, payment integrations, webhook support, and a playground for testing models. The backend is built with Go using the Gin web framework and GORM ORM, while the frontend is a React/TypeScript application built with Bun. The system implements a modular architecture with clear separation of concerns across controller, service, model, and relay layers.