$ show NoizAI/AudioX-Turbo
AudioX-Turbo is a unified framework for efficient anything-to-audio generation (text, video, audio conditions → audio) using Distribution Matching Distillation (DMD) adapted to flow matching and adversarial training. It supports six core tasks: Text-to-Audio, Text-to-Music, Video-to-Audio, Video-to-Music, Text+Video-to-Audio, and Text+Video-to-Music. The system includes training scripts, an interactive Gradio demo, and a modular architecture with interchangeable components (attention, bottlenecks, etc.) via a factory pattern. Key innovations enable high-quality 4-step audio generation, reducing computational cost by up to 25× compared to multi-step diffusion models.
The LLM identified 4 files worth highlighting for reuse:
Provides a centralized factory for creating models, pretransforms, bottlenecks, and other components from JSON configuration. Enables modular and configurable system architecture, allowing easy swapping of implementations without changing core logic.
Implements efficient local attention mechanisms that reduce computational complexity from O(n^2) to O(n*k) for sequence length n and window size k. This self-contained module can be integrated into any transformer-based model to improve efficiency for long sequences.
Contains a collection of bottleneck layers (VAE, VQ, RVQ, FSQ) that can be plugged into the model architecture. Each bottleneck is implemented as a self-contained module, enabling experimentation with different discrete or continuous latent representations.
Defines a `MultiLoss` class that combines multiple loss functions (e.g., L1, MSE, perceptual, adversarial) with configurable weights. This reusable utility simplifies the implementation of complex loss systems in training loops.