$ show pnnbao97/VieNeu-TTS
VieNeu-TTS is an advanced on-device Vietnamese text-to-speech library with instant voice cloning, supporting bilingual (Vietnamese/English) synthesis. The system provides a Python SDK (`vieneu`) with multiple backends: v3 Turbo (default, 48kHz, ONNX or PyTorch), v3 Nano (lightweight 24kHz, ONNX only), and legacy v1/v2 (deprecated). It includes a Gradio web UI (`vieneu-web`), a FastAPI streaming server, and a browser-based HTML/JS client. The core architecture is a factory pattern (`Vieneu(mode=...)`) that returns the appropriate engine, all inheriting from a shared base class. Inference can run entirely torch-free on CPU via ONNX Runtime, with optional GPU acceleration (CUDA) for batched generation and lower latency. Feature highlights include instant voice cloning from a 3β8 second reference clip using a speaker encoder (ONNX) and denoiser, inline emotion cues (`[cΖ°α»i]`, `[thα» dΓ i]`), frame-level streaming, SRT subtitle-to-speech, and LoRA fine-tuning for custom voices (requires ~6GB GPU). The system is deployed via Docker with CPU and GPU profiles, and has CI/CD pipelines for testing and PyPI publishing. The codebase is well-structured with clear separation of concerns, though it carries some legacy weight from older model versions and a few deprecated parameters. Other notable components include a sliding-window repetition penalty, a babble guard heuristic for short utterances, and a voice persistence system for user-defined presets.
The LLM identified 4 files worth highlighting for reuse:
Implements a sliding-window repetition history with FIFO eviction (`_ChannelWindow`) used to penalize repeated token sequences during autoregressive generation. The class is self-contained, isolated from the rest of the engine, and can be reused in any autoregressive text or audio generation system that requires repetition penalty. It exposes a clean interface for resetting, pushing, and checking the history, and supports both single-path and batched generation.
Provides a lazy-loaded ONNX inference wrapper for the speaker encoder (x-vector model). It handles model loading, input preprocessing, and embedding extraction from a reference audio clip. The module is fully decoupled from the rest of the TTS pipeline and can be extracted as a standalone speaker embedding service for any voice cloning or speaker verification task. It uses ONNX Runtime exclusively, with no PyTorch dependency.
Contains the babble guard heuristic (`babble_suspect`, `babble_prefer`) that detects early EOS or run-on artifacts in generated audio chunks and triggers re-generation with adjusted parameters. The logic is self-contained, operates on token count and log-probability statistics, and can be adapted to any autoregressive audio generation system that needs to detect and recover from truncated or looping outputs. Also includes reusable audio utilities like `join_audio_chunks`, `trim_and_fade`, and `pause_pad_samples`.
Implements a clean factory function `Vieneu(mode=...)` that returns the appropriate TTS engine class based on a string identifier (e.g., 'turbo', 'nano', 'standard', 'fast', 'remote'). It abstracts away the complex selection logic and allows users to switch backends with a single parameter. The pattern is well-structured and reusable for any multi-backend system where different implementations share a common interface (defined in `base.py`).