thehardwarestore.ai

This repository implements a low-latency, real-time speech-to-speech conversational AI system. It provides an OpenAI Realtime API-compatible server over WebSocket and WebRTC transports, along with a CLI tool for local, talk, and serve modes. The architecture is modular, with separate handler stages for Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM) reasoning, and Text-to-Speech (TTS), connected via typed queues. Multiple backends are supported for each stage, including local models (Whisper, Parakeet, Kokoro, Qwen3-TTS, MLX) and cloud APIs (OpenAI, vLLM, Hugging Face Inference Providers). A demo web application with HF OAuth, rate limiting, and audio worklets is included for browser-based interaction. The system is designed for production use, with emphasis on low latency through speculative turn handling, cancellation scopes, and turn latency instrumentation. Extensive test suites cover unit, integration, and protocol contract tests, including an evaluation harness for Big Bench Audio. While primarily an application, the repo also includes data/research components such as benchmarking scripts and a custom speech evaluation dataset.

Silero VAD is a pre-trained enterprise-grade Voice Activity Detector library that provides speech detection capabilities for audio signals. The system consists of a core PyTorch model (also available in ONNX format) with Python utilities for audio processing, speech timestamp extraction, and streaming applications. It supports 8kHz and 16kHz sampling rates and includes a VADIterator class for real-time processing. The repository offers multi-language examples (C++, C#, Go, Haskell, Java, Rust) demonstrating integration, along with tools for model tuning and sequence-based ONNX optimization for improved performance on long audio files.

Praat is a cross-platform desktop application for phonetic analysis, speech manipulation, and linguistic research. It provides a rich GUI with interactive editors for viewing and editing sounds, spectrograms, pitch, formants, text annotations, and more. The core architecture is a monolithic C++ application built on a custom object system (Thing/oo_* macros) with automatic serialization, a comprehensive preferences framework, and a scriptable interface. The codebase bundles many external libraries (GSL, eSpeak, FLAC, Opus, LAME, PortAudio, WhisperCPP, etc.) for numerical computing, audio I/O, speech synthesis, and machine learning. It includes modules for linear predictive coding (LPC), cepstral analysis, neural networks (FFNet), optimality theory grammar, multidimensional scaling, and EEG/ERP processing. While the code is well-structured and domain-specific, the heavy reliance on external libraries, custom object system, and platform-specific ifdefs introduce maintenance complexity and moderate technical drift.

Parakeet.cpp is a high-performance speech recognition inference library for NVIDIA's Parakeet model family (TDT-CTC, RNNT, Streaming EOU, Nemotron, Sortformer), written in C++ with C bindings. It provides offline and streaming transcription, speaker diarization, voice activity detection (VAD), beam search with optional n-gram language model fusion, phrase boosting, and timestamp extraction. The library is built on the axiom tensor library with Metal GPU acceleration for Apple Silicon, and supports audio input from WAV, FLAC, MP3, and OGG formats. The codebase includes a CLI tool, a Unix domain socket server example, and a rich set of examples covering all major features. While primarily an application/library for production ASR inference, it also contains research-oriented model conversion scripts (NeMo to safetensors) and a benchmark script comparing with PyTorch inference, giving it a hybrid character. The architecture is cleanly modular, with separate modules for audio I/O, VAD, model encoders/decoders, and decoding algorithms. The C API enables binding to Python, Swift, Rust, and other languages. Documentation and build system are well-maintained, and the repository is actively developed with recent commits.

ModelScope is a unified framework for model inference, training, and deployment across computer vision, NLP, audio, multi-modal, and scientific domains. It provides a registry-based architecture where models, pipelines, preprocessors, trainers, and metrics are registered by task and can be instantiated from configuration or directly via Python APIs. The framework integrates tightly with the ModelScope Hub for model and dataset distribution, supporting versioned model IDs and snapshot downloads. Key entry points include the `pipeline()` function for inference, `build_trainer` for training, and CLI tools for cache management and server deployment. The architecture is modular with well-defined base classes (TorchModel, Pipeline, Preprocessor, Metric, Hook) and a configuration system that drives component construction. External dependencies span PyTorch, TensorFlow, HuggingFace Transformers, MMDetection, Diffusers, and numerous domain-specific libraries. The codebase is large and actively developed, but shows signs of technical drift through duplicated research code, skipped tests, and legacy TensorFlow 1.x components.

VoxCPM is a tokenizer-free, multilingual text-to-speech (TTS) system developed by OpenBMB. It supports voice cloning, voice design, and controllable speech generation across 30 languages and 9 Chinese dialects, outputting 48kHz audio. The architecture is a diffusion-autoregressive model built on MiniCPM-4, comprising AudioVAE, Local Encoder, Text-Semantic LM, Residual Acoustic LM, and Local DiT. The repository provides multiple interfaces: a Gradio web application (app.py) for interactive use, a command-line interface (voxcpm.cli) for batch processing, a fine-tuning web UI (lora_ft_webui.py) for LoRA and full-parameter training, and Python APIs for integration. It also includes utilities for audio enhancement (ZipEnhancer), timestamp alignment, and text normalization. While primarily an application, it contains research-grade model definitions and training pipelines, making it a hybrid project with a strong application focus.

This repository is an offline speech-to-text application that uses the faster-whisper model to transcribe audio and video files into text, JSON, or SRT formats. It provides a web interface built with LayUI and a REST API (including an OpenAI-compatible endpoint) for easy integration. The application supports CPU and CUDA acceleration, caches loaded models for efficiency, and handles media conversion via FFmpeg. The architecture consists of a Flask web server served by gevent, with a background thread processing transcription tasks from a queue. Configuration is managed via an INI file, and utilities for FFmpeg execution, update checking, and browser launching are encapsulated in a helper module. The model caching mechanism avoids reloading large models for multiple requests, improving performance in a server setting.

MockingBird is a Chinese voice cloning system implementing the SV2TTS (Speaker Verification to Text-to-Speech) framework. It provides a complete pipeline for few-shot voice cloning: a speaker encoder extracts d-vector embeddings from reference audio, a synthesizer (Tacotron with Global Style Tokens or VITS) converts text to mel-spectrograms conditioned on the speaker embedding, and a vocoder (HiFi-GAN, Fre-GAN, or WaveRNN) generates waveforms. It also includes a PPG-based voice conversion path using a Conformer-based PPG extractor and a PPG2Mel converter. The system supports multiple Chinese datasets (aidatatang_200zh, magicdata, aishell3, data_aishell) and provides web UIs (Streamlit/FastAPI), a PyQt5 toolbox, and CLI tools for training and inference. The codebase is a fork of Real-Time-Voice-Cloning adapted for Mandarin with significant additions including VITS synthesizer, multiple vocoder options, emotion extraction via wav2vec2, and a modern web API layer. While the original author has moved to a commercial product (noiz.ai), the repository shows recent commits (2026) and maintains a large community (36k stars).