WhisperPlus is a Python library that provides enhanced speech processing capabilities built around Whisper models. It offers a modular pipeline architecture for speech-to-text transcription with optimizations like quantization and flash attention, text summarization (including long-text handling), speaker diarization, text-to-speech conversion, video auto-captioning, and retrieval-augmented generation for chatting with video content. The library integrates with Hugging Face Transformers, Apple MLX for optimized inference on Apple silicon, and various external services like YouTube downloading and vector databases for RAG functionality. The architecture follows a modular design with clearly separated pipelines for each functionality: core speech-to-text (whisper.py), summarization (summarization.py, long_text_summarization.py), text-to-speech (text2speech.py), diarization (whisper_diarize.py), auto-captioning (whisper_autocaption.py), and RAG chatbots (lancedb_chatbot.py, autollm_chatbot.py). Utility modules handle YouTube media downloading (download_utils.py) and text formatting (text_utils.py). The library provides both programmatic API access and a Gradio-based web interface (app.py) for interactive use, along with a CLI entry point (cli.py). Data flows typically begin with audio/video input (from files or YouTube URLs), proceed through specialized processing pipelines (transcription, summarization, etc.), and produce text, audio, or video outputs. For RAG features, transcripts are processed into vector stores enabling question-answering interactions. The library relies on established dependencies including torch, transformers, ffmpeg, pyannote.audio for diarization, and MLX-specific libraries for Apple silicon optimization, demonstrating practical integration of state-of-the-art models into an accessible toolkit.
Parakeet.cpp is a high-performance speech recognition inference library for NVIDIA's Parakeet model family (TDT-CTC, RNNT, Streaming EOU, Nemotron, Sortformer), written in C++ with C bindings. It provides offline and streaming transcription, speaker diarization, voice activity detection (VAD), beam search with optional n-gram language model fusion, phrase boosting, and timestamp extraction. The library is built on the axiom tensor library with Metal GPU acceleration for Apple Silicon, and supports audio input from WAV, FLAC, MP3, and OGG formats. The codebase includes a CLI tool, a Unix domain socket server example, and a rich set of examples covering all major features. While primarily an application/library for production ASR inference, it also contains research-oriented model conversion scripts (NeMo to safetensors) and a benchmark script comparing with PyTorch inference, giving it a hybrid character. The architecture is cleanly modular, with separate modules for audio I/O, VAD, model encoders/decoders, and decoding algorithms. The C API enables binding to Python, Swift, Rust, and other languages. Documentation and build system are well-maintained, and the repository is actively developed with recent commits.