$ show fishaudio/Bert-VITS2
Bert-VITS2 is a multilingual text-to-speech system based on VITS2 (Variational Inference Text-to-Speech with improved Transformer backbone). It supports Chinese, Japanese, and English synthesis with optional emotion conditioning via pre-trained models (WavLM, wav2vec2-emotion, CLAP). The codebase provides a complete pipeline: data preprocessing (resampling, text normalization, BERT feature extraction), multi-speaker training with distributed support, inference through a Gradio web UI or FastAPI server (hiyoriUI.py), and ONNX export for deployment. The architecture uses separate per-language BERT models for text encoding, a flow-based decoder, stochastic duration predictor, and multi-scale discriminators (including a WavLM-based discriminator). The repository includes numerous versioned subdirectories (oldVersion/, onnx_modules/) for backward compatibility with model versions 1.0.1 through 2.4, resulting in significant code duplication and maintenance overhead. The README explicitly states the project is no longer maintained and recommends the successor project Fish-Speech.
The LLM identified 5 files worth highlighting for reuse:
Implements a comprehensive Japanese grapheme-to-phoneme (G2P) pipeline using pyopenjtalk for morphological analysis and accent extraction, combined with custom functions (kata2phoneme, text2sep_kata, handle_long, align_tones) for converting Japanese text into phoneme sequences with accent information. This module is self-contained, language-specific, and can be extracted into other multilingual TTS or speech processing projects.
Provides a NumPy/Numba-accelerated implementation of monotonic alignment search (MAS), a key component in VITS2 for aligning text and audio representations during training. The implementation uses dynamic programming with O(T*N) complexity and is isolated from the rest of the codebase, making it reusable in any TTS system that requires text-audio alignment.
A lightweight language classifier that identifies whether a given text string is Chinese, Japanese, English, or a mixture, using character-level heuristics (Unicode ranges and known patterns) without external dependencies. It is used by the multilingual TTS pipeline to route text to the appropriate BERT model and G2P module. The component is small, well-defined, and easily extractable for any project needing basic character-based language detection.
Defines the Transformer encoder/decoder and multi-head attention modules used in the VITS2 backbone, including relative position embeddings, FFN, and residual connections. These modules are the core of the text encoder and are designed with flexibility (configurable hidden size, layers, heads). Although they import commons.py and modules.py, they represent a clean implementation of Transformer components that can be adapted for other sequence modeling tasks.
Provides a language-agnostic text cleaner that dispatches to language-specific cleaning functions (chinese_cleaners, japanese_cleaners, english_cleaners) based on text content. It handles punctuation normalization, whitespace trimming, and language-specific rules. The pattern of delegation with a registry is a clean, reusable design for multilingual text processing pipelines.