$ show gradio-app/fastrtc
FastRTC is a Python library that enables developers to build real-time audio/video streaming applications with minimal code. It provides a high-level `Stream` class that integrates with Gradio for automatic UI generation and supports both WebRTC and WebSocket protocols. The library includes built-in handlers for voice activity detection (VAD) via Silero, speech-to-text via Moonshine, text-to-speech via Kokoro/Cartesia, and turn-taking logic. It also offers TURN credential helpers for Cloudflare, Twilio, and Hugging Face. The architecture revolves around defining a handler function that processes media chunks, wrapping it with turn-taking decorators, and creating a Stream object that handles signaling, connection management, and UI rendering. The codebase is actively maintained with frequent commits and an extensive set of demos integrating with various LLMs and AI services.
The LLM identified 5 files worth highlighting for reuse:
Implements a generic turn-taking handler using Silero VAD. It detects pauses in user speech and automatically triggers the response handler. Configurable thresholds and interruption support make it reusable for any voice-based interactive application. The pattern of combining a VAD model with event-driven response is non-obvious and well-crafted.
Provides a reusable mixin for managing WebRTC peer connections, including offer/answer exchange, ICE candidate handling, and track setup. It encapsulates the complex WebRTC lifecycle and can be extracted to create custom Gradio components with WebRTC support. Demonstrates deep expertise in WebRTC internals.
Wraps the Silero VAD model with a clean Python interface, handling model download, caching, and inference. It provides configurable thresholds and can be reused in any Python project needing real-time voice activity detection.
Contains well-typed utility functions for audio conversion (float32/int16/bytes), aggregation, and async queue helpers (`wait_for_item`). These functions are self-contained, handle edge cases, and can be extracted for any audio processing pipeline.
TypeScript module with utility functions for WebRTC peer connection setup, ICE candidate handling, and data channel management. It is reusable in any web-based real-time communication project and demonstrates clean separation of WebRTC logic from UI components.