Handy is a free, open-source, offline speech-to-text desktop application built with Tauri (Rust backend + React/TypeScript frontend). It records microphone audio, transcribes using local ASR models (Whisper, Parakeet, etc.), and pastes the text into any active application via global shortcuts. The architecture is a monolith with a single-process Rust backend that manages audio capture, VAD (Silero), model loading (GGUF/ONNX), transcription engines (transcribe-cpp, transcribe-rs), clipboard paste, system tray, and recording overlays. The frontend is a React SPA that communicates with the backend via Tauri commands and events, providing a rich settings UI, model management, onboarding, history, and post-processing via OpenAI-compatible LLM APIs or Apple Intelligence. The codebase exhibits high craftsmanship with extensive use of RAII guards, lock-free streaming, state machines, platform-specific abstractions, and thorough error handling. It is actively maintained with frequent commits, a large community, and cross-platform builds for macOS, Windows, and Linux, including Nix support.
whisper.cpp is a high-performance C/C++ inference engine for OpenAI's Whisper automatic speech recognition (ASR) model, with additional support for a wide range of large language models (LLMs) via an integrated llama.cpp fork. The core library is dependency-free and uses the ggml tensor library to run on CPU, GPU (CUDA, Metal, Vulkan, OpenCL, SYCL, etc.), and specialized hardware (Apple Neural Engine, Hexagon DSP, etc.). It provides a clean C API, extensive language bindings (Go, Java, Ruby, JavaScript, Swift, etc.), and a rich set of example applications including a CLI transcriber, real-time microphone streaming, an HTTP server with OpenAI-compatible API, voice-controlled chess, and a talk-llama chatbot. The repository also includes model conversion scripts (PyTorch to ggml, CoreML, OpenVINO), quantization tools, and evaluation benchmarks (LibriSpeech, Earnings21). While primarily an application library, it contains research-oriented features like grammar-constrained decoding, beam search, and experimental model architectures. The codebase is actively maintained with a large community, though the sheer number of hardware backends and model variants introduces some maintenance complexity.
Moonshine Voice is an open-source AI toolkit for building real-time voice agents and applications with on-device speech-to-text, text-to-speech, and voice cloning capabilities. The system emphasizes privacy, low latency, and offline operation through a cross-platform C++ core with language bindings for Python, Swift, Android, and WebAssembly. It includes optimized models for various platforms (from microcontrollers to desktops), multilingual grapheme-to-phoneme conversion, speaker diarization, and embedding-based phrase matching. The architecture features modular components for audio processing, model inference, and language-specific rules, with examples demonstrating microphone transcription, streaming TTS, agent workflows, and embedded deployments on RP2350 microcontrollers.
This repository implements a low-latency, real-time speech-to-speech conversational AI system. It provides an OpenAI Realtime API-compatible server over WebSocket and WebRTC transports, along with a CLI tool for local, talk, and serve modes. The architecture is modular, with separate handler stages for Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM) reasoning, and Text-to-Speech (TTS), connected via typed queues. Multiple backends are supported for each stage, including local models (Whisper, Parakeet, Kokoro, Qwen3-TTS, MLX) and cloud APIs (OpenAI, vLLM, Hugging Face Inference Providers). A demo web application with HF OAuth, rate limiting, and audio worklets is included for browser-based interaction. The system is designed for production use, with emphasis on low latency through speculative turn handling, cancellation scopes, and turn latency instrumentation. Extensive test suites cover unit, integration, and protocol contract tests, including an evaluation harness for Big Bench Audio. While primarily an application, the repo also includes data/research components such as benchmarking scripts and a custom speech evaluation dataset.
RealtimeSTT is a production-grade real-time speech-to-text library and server. The core library provides `AudioToTextRecorder`, a facade that orchestrates microphone capture, voice activity detection (VAD), wake-word detection, and multiple ASR backends (faster-whisper, sherpa-onnx, whisper.cpp, etc.) for low-latency realtime and final transcription. It features a dual-lane architecture combining an ultrafast speculative model with a slower accurate model, merged via a sticky alignment algorithm, and a text stabilizer that commits only evidence-backed partials. The server component (`stt-server-production`) is a FastAPI/WebSocket application with a versioned protocol (`realtimestt.remote.v1`) supporting per-session ordered event delivery, fair inference queues, preview snapshots, resume, and HTTP transcription. External dependencies include PyTorch, torchaudio, faster-whisper, sherpa-onnx, and optional engines. The system is designed for voice assistants, dictation, and multi-user ASR services.
YouDub WebUI is a full-stack video localization platform that enables users to import videos, transcribe speech via Whisper, translate via OpenAI-compatible APIs, clone voices with VoxCPM2, separate audio with Demucs, align subtitles with Qwen3 Forced Aligner, and export dubbed videos or subtitles. The architecture is plugin-based (Cordis framework) with a Node.js/TypeScript host orchestrating Python subprocesses for ML inference, a Next.js web frontend, and native desktop apps for macOS (Swift) and Windows (C#). The system features a task engine with queuing, cancellation, retry, and partial rerun; a worker protocol using NDJSON over stdin/stdout; and a comprehensive setup service that downloads and verifies model weights. While primarily an application, it includes research-grade model consumption and detailed validation records documenting real-media runs with SHA-256 hashes and human listening feedback. The codebase is actively maintained with recent commits, CI/CD pipelines, and extensive unit tests for contracts, protocols, and permissions.
WhisperPlus is a Python library that provides enhanced speech processing capabilities built around Whisper models. It offers a modular pipeline architecture for speech-to-text transcription with optimizations like quantization and flash attention, text summarization (including long-text handling), speaker diarization, text-to-speech conversion, video auto-captioning, and retrieval-augmented generation for chatting with video content. The library integrates with Hugging Face Transformers, Apple MLX for optimized inference on Apple silicon, and various external services like YouTube downloading and vector databases for RAG functionality. The architecture follows a modular design with clearly separated pipelines for each functionality: core speech-to-text (whisper.py), summarization (summarization.py, long_text_summarization.py), text-to-speech (text2speech.py), diarization (whisper_diarize.py), auto-captioning (whisper_autocaption.py), and RAG chatbots (lancedb_chatbot.py, autollm_chatbot.py). Utility modules handle YouTube media downloading (download_utils.py) and text formatting (text_utils.py). The library provides both programmatic API access and a Gradio-based web interface (app.py) for interactive use, along with a CLI entry point (cli.py). Data flows typically begin with audio/video input (from files or YouTube URLs), proceed through specialized processing pipelines (transcription, summarization, etc.), and produce text, audio, or video outputs. For RAG features, transcripts are processed into vector stores enabling question-answering interactions. The library relies on established dependencies including torch, transformers, ffmpeg, pyannote.audio for diarization, and MLX-specific libraries for Apple silicon optimization, demonstrating practical integration of state-of-the-art models into an accessible toolkit.
SenseVoice is a production-grade speech understanding application built around the SenseVoiceSmall encoder-only model. It provides automatic speech recognition (ASR), language identification, speech emotion recognition (SER), and audio event detection (AED) via a non-autoregressive CTC pipeline. The Python codebase includes a FastAPI inference server, Gradio web UI, ONNX/Libtorch export utilities, a long-audio inference script with bounded memory, and a fine-tuning workflow backed by DeepSpeed. The repository also ships an independent C++ runtime under `runtime/llama.cpp/` that ports the same model to GGUF format for CPU/edge deployment without Python. While the core model (`model.py`) is a thin wrapper around the FunASR library and tightly coupled to its conventions, supporting utilities like CTC forced alignment, device resolution, and SER evaluation are well-isolated and reusable. The llama.cpp C++ components (VAD, SenseVoice decoder, and an OpenAI-compatible streaming server) are self-contained, architecturally clean, and validated against the PyTorch reference, making them the highest-value reusable assets. The project is actively maintained with recent commits, thorough tests, and CI/CD for both container builds and the C++ runtime.
This toolkit provides a command-line interface for transcribing long audio and video files using the Qwen-ASR API. It overcomes the API's 3-minute duration limit by splitting audio into chunks based on voice activity detection (VAD) and processing them in parallel. The toolkit includes reusable components for audio loading, VAD-based segmentation, and API interaction with retry logic and text post-processing. The main workflow involves loading media, applying VAD to detect speech segments, splitting audio into chunks respecting a target duration, transcribing chunks in parallel via the API, and combining results with optional SRT subtitle generation.
Dograh is an open-source, self-hostable voice AI platform for building, deploying, and monitoring voice agents. It provides a visual workflow builder (node-based graph editor) for designing call flows, with deep integration across telephony providers (Twilio, Plivo, Vonage, Telnyx, Cloudonix, Vobiz, Asterisk ARI), STT/TTS/LLM models (OpenAI, Deepgram, Google, Azure, Cartesia, ElevenLabs, etc.), and real-time audio pipelines via the Pipecat library. The backend is a FastAPI monolith with PostgreSQL, Redis, and object storage, complemented by separate worker services (ARQ for background tasks) and a campaign orchestrator for bulk outbound calling. The system supports multi-tenancy, call recording, knowledge base/RAG, QA analysis, embeddable chat widgets, and an MCP server for AI-assisted workflow management. SDKs for Python and TypeScript enable programmatic workflow creation and outbound call triggering. While the core is a full-stack application, it includes evaluation frameworks for STT providers and post-call QA analysis, giving it a hybrid character but dominated by production application concerns.
Parakeet.cpp is a high-performance speech recognition inference library for NVIDIA's Parakeet model family (TDT-CTC, RNNT, Streaming EOU, Nemotron, Sortformer), written in C++ with C bindings. It provides offline and streaming transcription, speaker diarization, voice activity detection (VAD), beam search with optional n-gram language model fusion, phrase boosting, and timestamp extraction. The library is built on the axiom tensor library with Metal GPU acceleration for Apple Silicon, and supports audio input from WAV, FLAC, MP3, and OGG formats. The codebase includes a CLI tool, a Unix domain socket server example, and a rich set of examples covering all major features. While primarily an application/library for production ASR inference, it also contains research-oriented model conversion scripts (NeMo to safetensors) and a benchmark script comparing with PyTorch inference, giving it a hybrid character. The architecture is cleanly modular, with separate modules for audio I/O, VAD, model encoders/decoders, and decoding algorithms. The C API enables binding to Python, Swift, Rust, and other languages. Documentation and build system are well-maintained, and the repository is actively developed with recent commits.
This repository is the engineering backbone for llamafile, a project by Mozilla.ai that packages large language model (LLM) inference engines into single, portable executables. It is primarily an application distribution and runtime platform, not a data research project. The core innovation is the use of Cosmopolitan Libc to create "Actually Portable Executables" (APEs) that run on Linux, macOS, Windows, and BSD without installation. The codebase integrates heavily patched versions of llama.cpp, whisper.cpp, stable-diffusion.cpp, and agent.cpp as submodules. It provides a unified CLI, TUI (chatbot), and HTTP server (OpenAI-compatible API) for interacting with these models. It features a sophisticated dynamic GPU backend loader (CUDA, ROCm, Vulkan, Metal), a custom high-performance CPU matrix multiplication library (TinyBLAS), integer quantized kernels (IQK), and a cross-platform sandboxing module (pledge/SECCOMP/Landlock). An agent framework (agentfile) allows for tool-calling workflows. The codebase demonstrates high engineering craftsmanship in specific areas, particularly the TinyBLAS CUDA replacement, the dynamic GPU backend abstraction, the state-machine syntax highlighting library, and the embedded readline replacement (Bestline). However, the project carries significant technical drift risk due to its heavy reliance on fast-moving upstream submodules and a complex, platform-specific build system. The patch management workflow is well-documented but inherently fragile.
This repository is an offline speech-to-text application that uses the faster-whisper model to transcribe audio and video files into text, JSON, or SRT formats. It provides a web interface built with LayUI and a REST API (including an OpenAI-compatible endpoint) for easy integration. The application supports CPU and CUDA acceleration, caches loaded models for efficiency, and handles media conversion via FFmpeg. The architecture consists of a Flask web server served by gevent, with a background thread processing transcription tasks from a queue. Configuration is managed via an INI file, and utilities for FFmpeg execution, update checking, and browser launching are encapsulated in a helper module. The model caching mechanism avoids reloading large models for multiple requests, improving performance in a server setting.
This repository is a well-established Python library (`SpeechRecognition`) that provides a unified interface for multiple speech recognition engines, including Google Speech, CMU Sphinx (PocketSphinx), Vosk, OpenAI Whisper (both local and API), Groq, Cohere, and several others. The architecture follows a facade pattern via a central `Recognizer` class, which handles audio capture (microphone, files) and delegates recognition to pluggable backends – each implemented as a standalone function in the `recognizers/` subpackage and attached to the `Recognizer` instance at import time. The library bundles FLAC encoder binaries for audio conversion, offers a CLI tool for model downloads, and includes comprehensive test suites with audio fixtures in multiple formats. While the code is well-structured and features clean abstractions (e.g., `AudioData` for format conversion, `OpenAICompatibleRecognizer` base class), it exhibits some technical drift: bundled FLAC binaries are outdated (v1.3.2 from 2014), it depends on `audioop` (deprecated in Python 3.13+), and several backend APIs are at risk of deprecation. The library is production-grade (v3.17.0) and widely used, but its reliance on hardcoded API keys and legacy workarounds for old Python versions suggests maintenance is ongoing but with moderate lag. Overall, it is a solid, reusable codebase with several components that could be extracted independently.
FastRTC is a Python library that enables developers to build real-time audio/video streaming applications with minimal code. It provides a high-level `Stream` class that integrates with Gradio for automatic UI generation and supports both WebRTC and WebSocket protocols. The library includes built-in handlers for voice activity detection (VAD) via Silero, speech-to-text via Moonshine, text-to-speech via Kokoro/Cartesia, and turn-taking logic. It also offers TURN credential helpers for Cloudflare, Twilio, and Hugging Face. The architecture revolves around defining a handler function that processes media chunks, wrapping it with turn-taking decorators, and creating a Stream object that handles signaling, connection management, and UI rendering. The codebase is actively maintained with frequent commits and an extensive set of demos integrating with various LLMs and AI services.
Wisp is a local-first, real-time meeting transcription desktop application built with Rust and SvelteKit, using Tauri v2 for cross-platform deployment (macOS Apple Silicon, Windows x64). It captures microphone and system audio, runs on-device automatic speech recognition (ASR) using multiple engines (whisper.cpp, sherpa-onnx, Apple Speech), provides live speaker-diarized transcripts with timestamps, and supports offline semantic search via local ONNX embedding models. The architecture is modular, with pluggable traits for audio sources, ASR engines, denoisers, echo cancellers, and model stores. It also offers optional cloud transcription (OpenAI, Gemini, Groq, etc.) and AI assist features. Although primarily an application, it integrates significant data engineering patterns for streaming audio processing, feature extraction (VAD, diarization), and on-device model management.
ASRT (Automatic Speech Recognition Toolkit) is a Chinese speech recognition system that provides both training and inference capabilities. It uses a deep CNN acoustic model with Connectionist Temporal Classification (CTC) and an N-gram language model based on a maximum entropy hidden Markov model. The system supports multiple backends (TensorFlow/Keras and experimental PyTorch) and can be trained on several Chinese speech corpora (THCHS30, ST-CMDS, etc.). It includes HTTP and gRPC servers for real-time inference, a speech recorder utility, and client libraries. The architecture is monolithic with separate scripts for training, evaluation, and serving. The codebase is functional but shows signs of technical drift, such as using deprecated Keras APIs (fit_generator) and hardcoded paths. The language model implementation is a notable highlight, offering a non-trivial statistical decoding approach. The repository is popular but has a single maintainer and a high number of open issues.
Voquill is a cross-platform voice dictation and AI transcription application that allows users to dictate into any application with real-time speech recognition, AI post-processing, and an agent mode for automated tasks. The desktop app (Tauri + Rust + React) provides a native overlay, global hotkeys, and multiple transcription backends (local Whisper, cloud providers, enterprise). The enterprise edition includes a gateway service with PostgreSQL, admin panel, OIDC SSO, and custom STT/LLM providers. Mobile apps (Flutter) with keyboard extensions provide dictation on iOS and Android. The CLI tool offers headless dictation and agent functionality. The architecture is a monorepo with shared packages for types, utilities, voice AI integrations, and agent loop, demonstrating a well-structured, multi-platform application with significant hybrid characteristics due to extensive data pipelines for transcription and model management.