$ show xinnan-tech/xiaozhi-esp32-server
This is the server platform for the XiaoZhi (小智) ESP32 voice-assistant ecosystem: a large multi-component monorepo that turns low-cost ESP32 hardware into an AI voice assistant. The core is a Python asyncio service (main/xiaozhi-server) that maintains WebSocket connections with devices and runs a real-time audio pipeline — VAD, ASR, intent/LLM orchestration, tool calling, and TTS — streaming audio back to the device. Device onboarding, OTA firmware updates, and the WebSocket handshake are partly in the Python server and partly in a Java Spring Boot management API (main/manager-api) backed by MySQL + Redis, which exposes a broad admin surface: users, devices, agents, model-provider configuration (LLM/ASR/TTS/VAD/memory/intent), RAGFlow-backed knowledge bases, voice clone, voiceprint, chat history, and MCP tool management. Two management frontends exist (a Vue 2 web console and a uni-app/Vue 3 mobile app), plus a Live2D digital-human web client and an offline theme generator for firmware assets. It is an application, not a research repository — no notebooks, data pipelines, or analytical datasets; the 'data' features (RAG ingestion, voice cloning, chat-history reporting, performance testers) are all operational application features.
The LLM identified 6 files worth highlighting for reuse:
A global cache manager supporting multiple eviction strategies (LRU, TTL, fixed-size) with typed caches and thread-safe access. Self-contained and config-driven; can be lifted into any Python service needing structured multi-tenant caching without modification.
Streaming PCM-to-Opus encode/decode utility with frame alignment and last-frame padding. Demonstrates non-obvious audio-engineering knowledge (frame boundaries, Opus packet semantics) and is a clean, isolated module reusable in any real-time audio project.
Timestamp-based audio frame pacing controller that schedules send operations for smooth playback, including pause/resume and pre-buffering. Implements a subtle real-time pacing problem without dragging in the rest of the server; extractable as a standalone rate limiter for streaming media.
Daily-rotating SHA-256 token authentication for MQTT gateway connections, with a ±1-day acceptance window and 401 retry handling. A small, well-contained security component that solves clock-skew-tolerant credential rotation — directly reusable in any gateway auth design.
A script that enforces locale-key parity across six translation files, failing CI when keys drift between languages. Solves the recurring i18n drift problem in a generic way; adaptable to any frontend with keyed translation dictionaries.
Abstract streaming TTS pipeline with text segmentation, slide-window word correction, and multi-format audio conversion (PCM/Opus). Demonstrates a clean base-class contract that dozens of TTS providers plug into via a config-driven factory; the segmentation/correction logic is reusable independently of any TTS vendor.