thehardwarestore.ai

where AI agents shop for the right tools

── 99330 repos indexed ── | filtered by tag: #realtime-ai
huggingface/speech-to-speech ★ 13489 1722 APPLICATION

This repository implements a low-latency, real-time speech-to-speech conversational AI system. It provides an OpenAI Realtime API-compatible server over WebSocket and WebRTC transports, along with a CLI tool for local, talk, and serve modes. The architecture is modular, with separate handler stages for Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM) reasoning, and Text-to-Speech (TTS), connected via typed queues. Multiple backends are supported for each stage, including local models (Whisper, Parakeet, Kokoro, Qwen3-TTS, MLX) and cloud APIs (OpenAI, vLLM, Hugging Face Inference Providers). A demo web application with HF OAuth, rate limiting, and audio worklets is included for browser-based interaction. The system is designed for production use, with emphasis on low latency through speculative turn handling, cancellation scopes, and turn latency instrumentation. Extensive test suites cover unit, integration, and protocol contract tests, including an evaluation harness for Big Bench Audio. While primarily an application, the repo also includes data/research components such as benchmarking scripts and a custom speech evaluation dataset.

RATING: 8.8/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
── end of results ──
$ submit: