$ show 2noise/ChatTTS
ChatTTS is a generative speech model application designed specifically for dialogue scenarios, functioning as a Text-to-Speech (TTS) library with integrated serving infrastructure. The core logic resides in the ChatTTS package, where `ChatTTS/core.py` defines the main inference interface. The architecture combines a GPT-based language model (`ChatTTS/model/gpt.py`) for token generation, a DVAE (`ChatTTS/model/dvae.py`) for audio compression, and Vocos as a vocoder. The system supports multiple speakers, prosodic control (laughter, pauses), and text normalization for Chinese and English. Deployment is handled through multiple entry points: a FastAPI-based API server (`examples/api/main.py`), a Gradio WebUI (`examples/web/webui.py`), and CLI tools (`examples/cmd/run.py`). A significant engineering highlight is the integration of vLLM (`ChatTTS/model/velocity/`) to optimize inference throughput for the GPT component. The repository includes robust asset management (`ChatTTS/utils/dl.py`) for model weights with SHA256 verification and ONNX export capabilities (`examples/onnx/exporter.py`) for cross-platform deployment. While the underlying model is research-grade (trained on 100k+ hours of audio), the repository structure prioritizes application delivery over experimental research. It provides a stable library interface with clear separation between model logic, audio processing (`tools/audio/`), and serving layers. The codebase demonstrates high craftsmanship in handling stochastic generation reproducibility (`tools/seeder/ctx.py`) and speaker embedding management (`ChatTTS/model/speaker.py`), making it a production-ready TTS solution rather than a raw research notebook collection.
The LLM identified 4 files worth highlighting for reuse:
Implements a high-throughput inference engine by wrapping vLLM components. It handles request scheduling, KV cache management, and block allocation for the GPT model. This component is reusable for any LLM-based service requiring optimized batch processing and memory management, demonstrating advanced integration of vLLM with custom model architectures.
Provides `TorchSeedContext` for managing random number generation states across PyTorch operations. It ensures reproducible audio generation by isolating seed context from global state. This is a reusable pattern for any stochastic ML application requiring deterministic outputs for debugging or consistent user experiences.
Encapsulates speaker embedding logic, including sampling, encoding, and decoding using `pybase16384` and `lzma`. It abstracts the complexity of speaker identity representation from the main inference pipeline. The pattern of managing speaker vectors as compressed, encoded assets is reusable for multi-speaker voice synthesis systems.
Manages model asset downloading with SHA256 verification against a static map (`sha256_map.json`). It handles network retries and integrity checks before loading weights. This pattern is reusable for any application distributing large binary assets where integrity and versioning are critical, preventing corruption or tampering.