This repository is a production-grade application for building real-time voice agents, primarily designed as an educational course but structured for actual deployment. It implements a complete voice call center system capable of receiving and making phone calls via Twilio, transcribing speech using various STT models (Moonshine, Whisper, Faster Whisper), reasoning with LLMs (Groq), searching property data using Superlinked and Qdrant, and generating speech using TTS models (Kokoro, Orpheus). The architecture is a modular monolith built with FastAPI, utilizing Docker for containerization and RunPod for GPU-accelerated model inference. While the repository includes Jupyter notebooks for educational purposes (Lesson 1-4), the core value lies in the `src/realtime_phone_agents` package, which encapsulates the agent logic, avatar system, and infrastructure integrations. The code demonstrates high craftsmanship in handling async audio streaming, managing external API dependencies, and implementing a flexible avatar registry system. It is suitable for engineers looking to build voice AI applications or understand the integration of LLMs with real-time audio pipelines.
libopus-wasm is a TypeScript library that provides WebAssembly bindings for libopus, enabling raw Opus packet encode/decode in browsers and Node.js without native dependencies. It exposes async factories (`createEncoder`, `createDecoder`) that lazily load a single-file WASM module, returning typed handles with methods for Int16 and Float32 PCM, batch operations, packet-loss concealment, in-band FEC, and a curated CTL passthrough. The library also ships a `@discordjs/opus`-compatible adapter (`libopus-wasm/discordjs`) for drop-in replacement. The codebase is well-structured with strict validation, deterministic cleanup via `Symbol.dispose`, and comprehensive test coverage. Build scripts compile libopus from source using Emscripten with SHA-256 verification, and the documentation site is generated from Markdown by a custom static site builder. The repository is actively maintained with recent commits, CI across Node 22–26, and no open issues.
Tachikoma is a production-grade Swift SDK/library that provides a unified typed interface to a broad spectrum of LLM providers: OpenAI (Chat Completions, the newer Responses API for GPT-5+ generations, and a ChatGPT Codex OAuth backend), Anthropic (Claude with streaming signed thinking blocks), Google Gemini, Mistral, Groq, Grok/xAI, Kimi, MiniMax, OpenRouter, Together, Replicate, Ollama, LM Studio, Azure OpenAI, and arbitrary OpenAI/Anthropic-compatible endpoints. Its public surface is deliberately modeled on the Vercel AI SDK: generateText/streamText, generateObject/streamObject with partial-JSON repair, embeddings, vision analysis, and multi-step agent tool-calling loops. Beyond generation, it ships composable stop conditions, response caching, retry policies, usage/cost accounting, reasoning-effort controls, and a provider feature-parity adapter that normalizes differences in system roles, vision, and tool limits across backends. The repo is strictly an application SDK in character: no datasets, notebooks, or data pipelines. The one data-like facet is its manually curated model capability catalog (context lengths, pricing, per-generation reasoning support), which is embedded as Swift enums rather than external registries. Architecturally it is a SwiftPM monorepo of five products: Tachikoma (core generation/provider layer), TachikomaAgent (conversations, dynamic tool registry, MCP tool binding), TachikomaAudio (transcription, TTS, and an OpenAI Realtime WebSocket voice client), TachikomaMCP (a client for Model Context Protocol servers over stdio/SSE/HTTP with full JSON Schema bridging), and TachikomaConfigCLI (auth/credential helper), plus two example CLIs (ai-cli, gpt5cli). All providers implement a common ModelProvider protocol with ProviderRequest/ProviderResponse and AsyncThrowingStream-based streaming, with shared SSE/NDJSON parsing funneled through OpenAICompatibleHelper and provider-specific adapters for Anthropic, Google, and Ollama. The code is Swift 6.2, heavily async/await, and covered by a substantial test suite (Swift Testing, mock URLProtocol fixtures, snapshot tests, serialized mutex-guarded tests for shared state), with CI across macOS and Linux. The codebase exhibits strong engineering practice: clean provider abstraction, genuinely reusable utilities (stream transforms, retry policies, TTL caches, typed JSON Schema views), modular product boundaries, and disciplined error modeling. Its principal weaknesses are the enormous provider surface that must be manually maintained (hardcoded capability matrices, context-length string heuristics like isOpus48/isSonnet5, static pricing tables in ModelCostCalculator), a handful of @unchecked Sendable singletons and NSLock-guarded global state, Linux/Apple streaming divergence (Linux buffers the full response instead of streaming bytes), some dead placeholder APIs (the legacy generate() returning a mock string), and a rapid feature-churn cadence visible in the changelog (GPT-5.6/6, Claude 5, reasoning replay). These are maintainability risks more than correctness defects; the tested behavior is consistently thorough. Overall this is a high-craft SDK that an engineer building AI application features in Swift would benefit from studying, particularly its OpenAI-compatible adapter pattern, cache-key determinism, and MCP schema translation.
This is the server platform for the XiaoZhi (小智) ESP32 voice-assistant ecosystem: a large multi-component monorepo that turns low-cost ESP32 hardware into an AI voice assistant. The core is a Python asyncio service (main/xiaozhi-server) that maintains WebSocket connections with devices and runs a real-time audio pipeline — VAD, ASR, intent/LLM orchestration, tool calling, and TTS — streaming audio back to the device. Device onboarding, OTA firmware updates, and the WebSocket handshake are partly in the Python server and partly in a Java Spring Boot management API (main/manager-api) backed by MySQL + Redis, which exposes a broad admin surface: users, devices, agents, model-provider configuration (LLM/ASR/TTS/VAD/memory/intent), RAGFlow-backed knowledge bases, voice clone, voiceprint, chat history, and MCP tool management. Two management frontends exist (a Vue 2 web console and a uni-app/Vue 3 mobile app), plus a Live2D digital-human web client and an offline theme generator for firmware assets. It is an application, not a research repository — no notebooks, data pipelines, or analytical datasets; the 'data' features (RAG ingestion, voice cloning, chat-history reporting, performance testers) are all operational application features.
This is a Python application and library for singing voice conversion and real-time voice changing, forked from so-vits-svc and rebuilt around PyTorch, PyTorch Lightning, and a Click-based CLI. The actual code provides two user-facing entry points: the `svc` CLI with commands for inference, realtime microphone conversion, dataset preprocessing, training, clustering, and cleanup, and the `svcg` PySimpleGUI desktop application for interactive and realtime inference. Inference loads a HuBERT content encoder from Hugging Face, extracts pitch using CREPE, DIO, Harvest, or Parselmouth, then runs a VITS-style SynthesizerTrn with either HiFi-GAN or MS/MB-iSTFT decoders conditioned on speaker embeddings. Despite fastapi being listed as a dependency, no HTTP API server exists in the source; the project is a desktop/CLI tool, not a web service.