$ show Aratako/Irodori-TTS
Irodori-TTS is a state-of-the-art Japanese text-to-speech system built on rectified flow diffusion over DACVAE audio latents. It provides a complete training and inference stack: `train.py` supports full fine-tuning, LoRA, speaker inversion, and MeanFlow distillation with distributed training and WSD scheduling; `infer.py` offers a rich CLI with CFG, Sway Sampling, duration scaling, and watermarking; `gradio_app.py` and `gradio_app_voicedesign.py` expose interactive UIs for reference-based cloning and caption-conditioned voice design. The architecture is modular (`irodori_tts` package) with attention dispatch (FA3/SDPA), a hybrid Muon+AdamW optimizer, codec abstraction, and caching-friendly `InferenceRuntime`. While highly engineered for production use (INT4/INT8 quantization, safetensors, HuggingFace Hub integration), it remains a research-oriented codebase with experimental sampling algorithms and many YAML configs for different model sizes/versions. The code is clean and well-factored, though `train.py` is a large monolith and external dependencies (DACVAE, silentcipher) are pinned to specific commits, creating ecosystem fragility.
The LLM identified 4 files worth highlighting for reuse:
`PretrainedTextTokenizer` is a thin, generic wrapper around HuggingFace tokenizers that enforces right-padding and optional BOS for TTS conditioning. It is fully self-contained, requiring only `transformers`, and can be dropped into any sequence-to-sequence or generative audio project needing consistent text tokenization. Demonstrates a clean adapter pattern with minimal coupling.
`context_attention` and the `ContextAttentionPlan` abstraction provide a high-performance cross-attention mechanism that dynamically dispatches to FlashAttention-3 varlen kernels or falls back to cuDNN-prioritized SDPA. The design isolates kernel selection from the model, making it reusable in any transformer-based model that needs efficient packed-sequence attention. This is a non-obvious, production-grade pattern for GPU-accelerated attention.
`_ManifestIndex` is a cached byte-offset index for JSONL datasets that enables O(1) random access and speaker grouping without loading the full file into memory. It is a self-contained utility (uses only standard Python and `orjson`-style JSON internals) and can be extracted to accelerate any large JSONL corpus processing. The pattern of building an in-memory index once and reusing it is a classic data-engineering optimization.
`MuonWithAuxAdamW` implements a hybrid optimizer that applies Muon updates to matrix parameters and AdamW to embeddings/biases, with weight-decay masking and pretrained-backbone learning-rate separation. It is a self-contained module (only depends on torch) and showcases a modern optimizer pattern that can be reused in any training pipeline where different parameter groups benefit from different update rules. The scheduler coupling is clean and framework-agnostic.