$ show KoljaB/RealtimeSTT
RealtimeSTT is a production-grade real-time speech-to-text library and server. The core library provides `AudioToTextRecorder`, a facade that orchestrates microphone capture, voice activity detection (VAD), wake-word detection, and multiple ASR backends (faster-whisper, sherpa-onnx, whisper.cpp, etc.) for low-latency realtime and final transcription. It features a dual-lane architecture combining an ultrafast speculative model with a slower accurate model, merged via a sticky alignment algorithm, and a text stabilizer that commits only evidence-backed partials. The server component (`stt-server-production`) is a FastAPI/WebSocket application with a versioned protocol (`realtimestt.remote.v1`) supporting per-session ordered event delivery, fair inference queues, preview snapshots, resume, and HTTP transcription. External dependencies include PyTorch, torchaudio, faster-whisper, sherpa-onnx, and optional engines. The system is designed for voice assistants, dictation, and multi-user ASR services.
The LLM identified 8 files worth highlighting for reuse:
Implements a deterministic algorithm for stabilizing realtime ASR partials: accumulates character/space/punctuation evidence, rejects outliers via SequenceMatcher similarity, adopts branches, and commits only when configurable thresholds are met. Reusable as a generic streaming-text stabilization primitive.
StickyRealtimeTranscriptionMerger combines a slow authoritative transcription stream with an ultrafast speculative stream using a 4-word fuzzy anchor. Never publishes unanchored ultrafast text, handles corrections and resume provenance. A reusable pattern for merging two latency-differentiated ASR outputs.
Auto-selects the optimal Silero VAD backend (raw ONNX if available, else ONNX, else PyTorch CPU) with legacy boolean compatibility. Encapsulates model loading and inference in a clean function, reusable across projects needing VAD.
Provides robust shutdown orchestration for multiprocessing workers: bounded joins, forceful terminate/kill, shared-resource close deduplication, and error logging. A reusable pattern for safely tearing down concurrent systems.
Handles external audio input: PCM16↔float conversion, stereo averaging, resample_poly, peak normalization, and realtime file feeding with pacing. Self-contained utilities for audio preprocessing, reusable in any audio pipeline.
Defines ModelManifest dataclass with archive size, SHA-256, expected file list, and per-file metadata. Used by the installer for verification. A clean, reusable pattern for pinning and verifying model distributions.
Downloads, verifies SHA-256, and atomically extracts sherpa-onnx model archives. Supports resume via .part files, offline cache reuse, and manifest-based verification. A reusable installer pattern for large model assets.
Implements the production-grade WebSocket/HTTP ASR server with versioned protocol, per-session ordered event delivery (OrderedConnectionManager), fair inference queues, preview snapshots, resume, and cancellation. The protocol design and delivery guarantees are reusable for any streaming event server.