Piper is a fast, local neural text-to-speech engine that uses VITS models exported to ONNX for inference. The repository provides a comprehensive application stack: a Python package (`piper-tts`) with a clean API for voice synthesis, a CLI tool, an HTTP server with a web interface (including viseme animation), and a C/C++ shared library (`libpiper`) for embedding in other applications. It also includes a full training pipeline using PyTorch Lightning, allowing users to fine-tune or train new voices from scratch, with support for multi-speaker models, custom phonemes, and VAD silence trimming. The architecture is modular: phonemization is handled by language-specific backends (espeak-ng, Chinese pinyin/g2pW, Hebrew Nakdimon, Japanese OpenJTalk, Arabic tashkeel), and the core inference engine uses ONNX Runtime for efficient, cross-platform execution. The codebase is well-structured with clear separation of concerns, extensive testing, and CI/CD pipelines that build wheels for all major platforms. Although primarily an application, the training code (VITS model implementations, dataset caching, monotonic alignment Cython) exhibits data/research characteristics, making it a hybrid project. The repository is actively maintained, with a responsive issue tracker and regular updates, though the last commit date appears anomalous.
Fish Speech is a production-grade multilingual text-to-speech system with a dual-autoregressive transformer architecture (4B parameter slow AR for temporal semantics, 400M fast AR for residual codebooks). It supports real-time voice cloning from short reference audio (10-30s), multi-speaker multi-turn generation, and fine-grained prosody/emotion control via natural language tags. The system is deployed through multiple interfaces: a Gradio web UI, a React-based modern web UI, a production ASGI API server (Kui) with streaming support, and command-line tools for inference and batch processing. Its architecture is modular: a TTSInferenceEngine orchestrates reference loading, LLM inference, and VQ decoding; an RVQ audio codec (10 codebooks, ~21 Hz frame rate) compresses audio; and the dual-AR model is structurally isomorphic to standard LLMs, enabling reuse of inference optimizations like paged KV cache and RadixAttention. The repository also includes training infrastructure (Lightning + Hydra) with LoRA fine-tuning, protobuf-based dataset pipelines, and a comprehensive i18n framework. While primarily a deployable application, it retains strong research character with published papers, benchmark results, and custom model implementations.
Amphion is a comprehensive open-source research toolkit for audio, music, and speech generation, developed by OpenMMLab. It covers multiple generative tasks including text-to-speech (TTS), singing voice conversion (SVC), voice conversion (VC), text-to-audio (TTA), neural audio codecs, and vocoders. The codebase is organized around config-driven experiments (JSON + shell scripts) with modular base classes for trainers, datasets, and inference. It ingests a wide variety of public datasets (LJSpeech, LibriTTS, VCTK, M4Singer, Emilia, etc.) and provides extensive preprocessing pipelines for feature extraction (mel-spectrograms, F0, energy, phonetic alignments, self-supervised features from HuBERT/Whisper). The primary research focus is on high-quality, zero-shot, and controllable generation using architectures like diffusion models, flow matching, GANs, autoregressive transformers, and vector-quantized codecs. The repository also includes evaluation metrics (F0, energy, intelligibility, speaker similarity) and visualization tools (SingVisio). While primarily a research codebase, it includes inference scripts and Gradio demos for practical use. External dependencies include PyTorch, HuggingFace Transformers, WeNet, Descript Audio Codec, and various pretrained models.
GPT-SoVITS is a production-grade text-to-speech and voice conversion system with a Gradio-based WebUI and FastAPI REST API. It supports zero-shot voice cloning from a 5-second reference audio and few-shot fine-tuning with 1 minute of data. The system integrates multiple model versions (v1–v4, v2Pro, v2ProPlus) with autoregressive (GPT-style) and non-autoregressive (SoVITS/CFM) components, plus a BigVGAN vocoder. It includes a full pipeline for dataset preparation (voice separation, ASR, text labeling), training (with DDP, LoRA, DPO), and inference (streaming, parallel, ONNX/TorchScript export). The codebase is heavily modular with reusable text frontends for Chinese, Japanese, Korean, Cantonese, and English, as well as audio processing tools (UVR5, BS-Roformer, AP-BWE). Despite its breadth, the code exhibits significant technical debt: version sprawl, hardcoded paths, global state, and experimental hacks. It is actively maintained with frequent bug fixes and feature additions.
This is a Python application and library for singing voice conversion and real-time voice changing, forked from so-vits-svc and rebuilt around PyTorch, PyTorch Lightning, and a Click-based CLI. The actual code provides two user-facing entry points: the `svc` CLI with commands for inference, realtime microphone conversion, dataset preprocessing, training, clustering, and cleanup, and the `svcg` PySimpleGUI desktop application for interactive and realtime inference. Inference loads a HuBERT content encoder from Hugging Face, extracts pitch using CREPE, DIO, Harvest, or Parselmouth, then runs a VITS-style SynthesizerTrn with either HiFi-GAN or MS/MB-iSTFT decoders conditioned on speaker embeddings. Despite fastapi being listed as a dependency, no HTTP API server exists in the source; the project is a desktop/CLI tool, not a web service.
Bert-VITS2 is a multilingual text-to-speech system based on VITS2 (Variational Inference Text-to-Speech with improved Transformer backbone). It supports Chinese, Japanese, and English synthesis with optional emotion conditioning via pre-trained models (WavLM, wav2vec2-emotion, CLAP). The codebase provides a complete pipeline: data preprocessing (resampling, text normalization, BERT feature extraction), multi-speaker training with distributed support, inference through a Gradio web UI or FastAPI server (hiyoriUI.py), and ONNX export for deployment. The architecture uses separate per-language BERT models for text encoding, a flow-based decoder, stochastic duration predictor, and multi-scale discriminators (including a WavLM-based discriminator). The repository includes numerous versioned subdirectories (oldVersion/, onnx_modules/) for backward compatibility with model versions 1.0.1 through 2.4, resulting in significant code duplication and maintenance overhead. The README explicitly states the project is no longer maintained and recommends the successor project Fish-Speech.