$ account CorentinJ
This is an implementation of SV2TTS (Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis) with a real-time vocoder, structured as a three-stage deep learning pipeline: speaker encoder, Tacotron-based synthesizer, and WaveRNN vocoder. The system clones a voice from a few seconds of reference audio and then synthesizes arbitrary text in that voice. The architecture is a monolithic Python application with CLI and PyQt5 GUI entry points. The encoder module implements a GE2E-loss LSTM speaker encoder trained on VoxCeleb/LibriSpeech. The synthesizer module is a Tacotron variant conditioned on speaker embeddings. The vocoder module is a modified WaveRNN (fatchord version) that generates audio in real-time. All three stages have independent training scripts and inference interfaces. The codebase was a landmark open-source voice cloning project but is now significantly outdated. It pins Python 3.9, PyTorch 1.10, and uses deprecated APIs (scipy.ndimage.morphology.binary_dilation, visdom, webrtcvad). The synthesizer uses a pure-Python instead of CUDA-optimized attention mechanism. The deepmind_version.py vocoder references non-existent utils modules, suggesting incomplete refactoring. The CI pipeline only runs import smoke tests.