This repository is a Home Assistant custom component that adds OpenAI-compatible text-to-speech to a HA instance. It is distributed as an integration for HACS and registered as a config-flow service integration. In production it exposes TTS entities, an `openai_tts.say` service that announces text to media players with automatic volume save/restore and optional chimes, and an `openai_tts.set_api_key` service for credential rotation. The architecture is a HA integration split into focused modules: config flow and profile subentries, a streaming TTS engine, a sentence-level audio pipelining layer, an ffmpeg-based loudness pipeline, push-driven API health tracking, cache and validation logic, repairs, diagnostics, and a large announcement orchestration module. Provider support is driven by declarative presets for OpenAI, Mistral, Groq, Lemonfox, Kokoro, Chatterbox, and custom OpenAI-compatible endpoints, and the code intentionally modern-targets HA's streaming TTS contract rather than retaining legacy fallback paths.
This repository implements a low-latency, real-time speech-to-speech conversational AI system. It provides an OpenAI Realtime API-compatible server over WebSocket and WebRTC transports, along with a CLI tool for local, talk, and serve modes. The architecture is modular, with separate handler stages for Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM) reasoning, and Text-to-Speech (TTS), connected via typed queues. Multiple backends are supported for each stage, including local models (Whisper, Parakeet, Kokoro, Qwen3-TTS, MLX) and cloud APIs (OpenAI, vLLM, Hugging Face Inference Providers). A demo web application with HF OAuth, rate limiting, and audio worklets is included for browser-based interaction. The system is designed for production use, with emphasis on low latency through speculative turn handling, cancellation scopes, and turn latency instrumentation. Extensive test suites cover unit, integration, and protocol contract tests, including an evaluation harness for Big Bench Audio. While primarily an application, the repo also includes data/research components such as benchmarking scripts and a custom speech evaluation dataset.
aspeak is a text-to-speech client for Azure TTS API, written in Rust. It provides a CLI, a Rust library, and Python bindings. The core functionality synthesizes speech from plain text or SSML using either the REST or WebSocket API. The architecture is modular: the library defines two synthesizer types (`RestSynthesizer` and `WebsocketSynthesizer`) with a unified trait, and a CLI wraps these for command-line use. The Python binding exposes a `SpeechService` class. The codebase demonstrates solid Rust engineering with builder patterns, comprehensive error handling, and support for proxies, custom audio formats, and rich SSML options. It is well-tested with unit and integration tests, has CI/CD pipelines, and is actively maintained.
ESPnet is an end-to-end speech processing toolkit built on PyTorch, covering a wide range of tasks: automatic speech recognition (ASR), text-to-speech (TTS), speech translation (ST), speech enhancement (SE), speaker recognition, spoken language understanding (SLU), singing voice synthesis (SVS), speech language modeling (SpeechLM), and more. The repository contains both a core Python library (`espnet2/`, `espnet3/`) and a large collection of reproducible recipes (`egs2/`, `egs3/`) for dozens of public corpora and research challenges. The architecture is modular: each task defines its own model, loss, training, and inference via a `Task` class, with abstract base classes for encoders, decoders, frontends, tokenizers, and losses. Configuration is YAML-driven, and the training pipeline uses PyTorch Lightning (in ESPnet3) or a custom trainer (ESPnet2). The system provides CLI tools (`espnet` with subcommands like `transcribe`, `synthesize`, `enhance`, `translate`) and a Python API (`Speech2Text.from_pretrained()`) for inference. It also includes Docker images for deployment and Hugging Face Spaces demos. The repository is hybrid: it is primarily a software framework/library (application) with a strong research character, as evidenced by the extensive recipe collection, benchmark results, and support for state-of-the-art models (Conformer, HuBERT, Whisper, VALL-E). The recipe system follows Kaldi-style data preparation and stage-based pipelines, enabling reproducible experimentation. External dependencies include PyTorch, torchaudio, sentencepiece, and optional components like flash-attn, fairseq, and s3prl. The codebase is actively maintained with CI across multiple OS and Python versions, unit tests, and integration tests.
VieNeu-TTS is an advanced on-device Vietnamese text-to-speech library with instant voice cloning, supporting bilingual (Vietnamese/English) synthesis. The system provides a Python SDK (`vieneu`) with multiple backends: v3 Turbo (default, 48kHz, ONNX or PyTorch), v3 Nano (lightweight 24kHz, ONNX only), and legacy v1/v2 (deprecated). It includes a Gradio web UI (`vieneu-web`), a FastAPI streaming server, and a browser-based HTML/JS client. The core architecture is a factory pattern (`Vieneu(mode=...)`) that returns the appropriate engine, all inheriting from a shared base class. Inference can run entirely torch-free on CPU via ONNX Runtime, with optional GPU acceleration (CUDA) for batched generation and lower latency. Feature highlights include instant voice cloning from a 3β8 second reference clip using a speaker encoder (ONNX) and denoiser, inline emotion cues (`[cΖ°α»i]`, `[thα» dΓ i]`), frame-level streaming, SRT subtitle-to-speech, and LoRA fine-tuning for custom voices (requires ~6GB GPU). The system is deployed via Docker with CPU and GPU profiles, and has CI/CD pipelines for testing and PyPI publishing. The codebase is well-structured with clear separation of concerns, though it carries some legacy weight from older model versions and a few deprecated parameters. Other notable components include a sliding-window repetition penalty, a babble guard heuristic for short utterances, and a voice persistence system for user-defined presets.
Irodori-TTS is a state-of-the-art Japanese text-to-speech system built on rectified flow diffusion over DACVAE audio latents. It provides a complete training and inference stack: `train.py` supports full fine-tuning, LoRA, speaker inversion, and MeanFlow distillation with distributed training and WSD scheduling; `infer.py` offers a rich CLI with CFG, Sway Sampling, duration scaling, and watermarking; `gradio_app.py` and `gradio_app_voicedesign.py` expose interactive UIs for reference-based cloning and caption-conditioned voice design. The architecture is modular (`irodori_tts` package) with attention dispatch (FA3/SDPA), a hybrid Muon+AdamW optimizer, codec abstraction, and caching-friendly `InferenceRuntime`. While highly engineered for production use (INT4/INT8 quantization, safetensors, HuggingFace Hub integration), it remains a research-oriented codebase with experimental sampling algorithms and many YAML configs for different model sizes/versions. The code is clean and well-factored, though `train.py` is a large monolith and external dependencies (DACVAE, silentcipher) are pinned to specific commits, creating ecosystem fragility.
VoxCPM is a tokenizer-free, multilingual text-to-speech (TTS) system developed by OpenBMB. It supports voice cloning, voice design, and controllable speech generation across 30 languages and 9 Chinese dialects, outputting 48kHz audio. The architecture is a diffusion-autoregressive model built on MiniCPM-4, comprising AudioVAE, Local Encoder, Text-Semantic LM, Residual Acoustic LM, and Local DiT. The repository provides multiple interfaces: a Gradio web application (app.py) for interactive use, a command-line interface (voxcpm.cli) for batch processing, a fine-tuning web UI (lora_ft_webui.py) for LoRA and full-parameter training, and Python APIs for integration. It also includes utilities for audio enhancement (ZipEnhancer), timestamp alignment, and text normalization. While primarily an application, it contains research-grade model definitions and training pipelines, making it a hybrid project with a strong application focus.
Amphion is a comprehensive open-source research toolkit for audio, music, and speech generation, developed by OpenMMLab. It covers multiple generative tasks including text-to-speech (TTS), singing voice conversion (SVC), voice conversion (VC), text-to-audio (TTA), neural audio codecs, and vocoders. The codebase is organized around config-driven experiments (JSON + shell scripts) with modular base classes for trainers, datasets, and inference. It ingests a wide variety of public datasets (LJSpeech, LibriTTS, VCTK, M4Singer, Emilia, etc.) and provides extensive preprocessing pipelines for feature extraction (mel-spectrograms, F0, energy, phonetic alignments, self-supervised features from HuBERT/Whisper). The primary research focus is on high-quality, zero-shot, and controllable generation using architectures like diffusion models, flow matching, GANs, autoregressive transformers, and vector-quantized codecs. The repository also includes evaluation metrics (F0, energy, intelligibility, speaker similarity) and visualization tools (SingVisio). While primarily a research codebase, it includes inference scripts and Gradio demos for practical use. External dependencies include PyTorch, HuggingFace Transformers, WeNet, Descript Audio Codec, and various pretrained models.
This is a Python application and library for singing voice conversion and real-time voice changing, forked from so-vits-svc and rebuilt around PyTorch, PyTorch Lightning, and a Click-based CLI. The actual code provides two user-facing entry points: the `svc` CLI with commands for inference, realtime microphone conversion, dataset preprocessing, training, clustering, and cleanup, and the `svcg` PySimpleGUI desktop application for interactive and realtime inference. Inference loads a HuBERT content encoder from Hugging Face, extracts pitch using CREPE, DIO, Harvest, or Parselmouth, then runs a VITS-style SynthesizerTrn with either HiFi-GAN or MS/MB-iSTFT decoders conditioned on speaker embeddings. Despite fastapi being listed as a dependency, no HTTP API server exists in the source; the project is a desktop/CLI tool, not a web service.