This repository implements a low-latency, real-time speech-to-speech conversational AI system. It provides an OpenAI Realtime API-compatible server over WebSocket and WebRTC transports, along with a CLI tool for local, talk, and serve modes. The architecture is modular, with separate handler stages for Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM) reasoning, and Text-to-Speech (TTS), connected via typed queues. Multiple backends are supported for each stage, including local models (Whisper, Parakeet, Kokoro, Qwen3-TTS, MLX) and cloud APIs (OpenAI, vLLM, Hugging Face Inference Providers). A demo web application with HF OAuth, rate limiting, and audio worklets is included for browser-based interaction. The system is designed for production use, with emphasis on low latency through speculative turn handling, cancellation scopes, and turn latency instrumentation. Extensive test suites cover unit, integration, and protocol contract tests, including an evaluation harness for Big Bench Audio. While primarily an application, the repo also includes data/research components such as benchmarking scripts and a custom speech evaluation dataset.
Dograh is an open-source, self-hostable voice AI platform for building, deploying, and monitoring voice agents. It provides a visual workflow builder (node-based graph editor) for designing call flows, with deep integration across telephony providers (Twilio, Plivo, Vonage, Telnyx, Cloudonix, Vobiz, Asterisk ARI), STT/TTS/LLM models (OpenAI, Deepgram, Google, Azure, Cartesia, ElevenLabs, etc.), and real-time audio pipelines via the Pipecat library. The backend is a FastAPI monolith with PostgreSQL, Redis, and object storage, complemented by separate worker services (ARQ for background tasks) and a campaign orchestrator for bulk outbound calling. The system supports multi-tenancy, call recording, knowledge base/RAG, QA analysis, embeddable chat widgets, and an MCP server for AI-assisted workflow management. SDKs for Python and TypeScript enable programmatic workflow creation and outbound call triggering. While the core is a full-stack application, it includes evaluation frameworks for STT providers and post-call QA analysis, giving it a hybrid character but dominated by production application concerns.