This repository provides the infrastructure for the Anticipatory Music Transformer, a research project for music generation using autoregressive transformers with anticipatory infilling. The codebase covers the full pipeline: preprocessing of MIDI files (Lakh MIDI dataset) into a compound token representation, tokenization into arrival-time or interarrival-time sequences with optional augmentation for anticipation training, and sampling from trained models using custom generation logic. It also includes scripts for human evaluation reproduction and model benchmarking. While primarily a research codebase for a specific paper, the library components in the `anticipation` package (particularly the conversion, ops, and sampling modules) are well-structured and could be reused in other music tokenization or generation projects. The repository does not include the training loop itself but provides configuration files for use with the Levanter training framework.
ACE-Step is a production-grade music generation system that implements a flow-matching diffusion model (ACEStepTransformer2D) with a custom DCAE autoencoder and vocoder. It provides multiple deployment options: a Gradio UI, CLI inference, and FastAPI REST endpoints. The system supports text-to-music, lyrics-conditioned generation, audio-to-audio editing, LoRA fine-tuning, and multilingual lyrics (17+ languages). Architecturally, it extends Hugging Face diffusers with custom attention processors (LiteLA linear attention), schedulers (Euler/Heun/PingPong flow matching), and a Conformer-based lyric encoder. It includes CPU offloading, overlapped decode, and memory optimizations for GPU-constrained environments. While primarily an application, the repository also contains training scripts (PyTorch Lightning) and dataset conversion utilities, reflecting its hybrid research-production nature.
ChatRWKV is a collection of inference tools, demos, and a pip package for the RWKV language model family (v4-v7), a 100% RNN architecture that rivals transformers in performance. The repository provides interactive chat scripts (chat.py, v2/chat.py), API demos (API_DEMO_CHAT.py), model conversion utilities, state merging, music generation (MIDI/ABC), and benchmarking scripts. The core RWKV model class (rwkv_pip_package/src/rwkv/model.py) supports flexible deployment strategies (CPU/GPU, fp16/fp32/bf16, int8 quantization, streaming) and includes custom CUDA kernels for efficient WKV operations. The tokenizer module (tokenizer/rwkv_tokenizer.py) offers a fast trie-based byte-level tokenizer alongside a reference implementation. While the repository is primarily a research/demo project with many ad-hoc scripts and hardcoded paths, the pip package and core components are well-crafted and demonstrate significant engineering skill in GPU optimization and model inference. The codebase is actively maintained and supports the latest RWKV-7 architecture.
Audiocraft is a research codebase from Meta (Facebook Research) for state-of-the-art audio generation, including MusicGen, AudioGen, MAGNeT, JASCO, MultiBand Diffusion, and AudioSeal watermarking. It provides training and inference pipelines for text-to-music, text-to-sound, music continuation, style transfer, and symbolic conditioning (chords, drums, melody). The repository is primarily a research library with extensive configuration-driven training (Hydra/Dora), evaluation metrics (FAD, KLD, CLAP, ViSQOL), and interactive Gradio demos. It is not a production application but a collection of models, solvers, and utilities for audio generation research. The codebase is well-structured with modular components (conditioners, codebook patterns, audio effects) and includes unit tests, but relies on internal datasets and cluster paths that limit full reproducibility. Pretrained models are available via HuggingFace Hub for inference.
GeekAI is a full-stack, production-grade AI creation platform providing chat, image generation (MidJourney, DALL-E, Jimeng, Stable Diffusion), video generation (Sora, Veo, Keling, MiniMax, Wan, Doubao), music generation (Suno), and PPT creation. The system features a Go + Gin backend with MySQL/Redis persistence and a Vue 3 (Element Plus/Vant) frontend supporting both desktop and mobile interfaces. It includes a complete monetization framework with power credits, multiple payment gateways (Alipay, WeChat Pay, EPay), user invite/reward system, and an administrative dashboard for managing users, models, API keys, and system configurations. Architecturally, the backend follows a layered pattern with handlers, services, and stores. Key design patterns include an adapter pattern for video generation providers (allowing easy addition of new AI services), a factory pattern for OSS uploaders (local, MinIO, Qiniu, Aliyun, Tencent), and an abstraction for payment integrations. Async tasks (e.g., image generation, video creation) are handled via Redis-based queues with polling for status updates. The frontend uses Pinia stores for state management and includes reusable UI components like custom tabs, file uploaders, and parameter builders. The codebase demonstrates solid engineering with moderate code quality. It has several genuinely reusable patterns (video adapter, upload manager, payment service interface) but also exhibits duplication between desktop and mobile views, some hardcoded configuration, and reliance on many third-party SDKs that may drift over time. While not exceptionally polished, it is a fully functional commercial-grade platform suitable for deployment and customization.
Amphion is a comprehensive open-source research toolkit for audio, music, and speech generation, developed by OpenMMLab. It covers multiple generative tasks including text-to-speech (TTS), singing voice conversion (SVC), voice conversion (VC), text-to-audio (TTA), neural audio codecs, and vocoders. The codebase is organized around config-driven experiments (JSON + shell scripts) with modular base classes for trainers, datasets, and inference. It ingests a wide variety of public datasets (LJSpeech, LibriTTS, VCTK, M4Singer, Emilia, etc.) and provides extensive preprocessing pipelines for feature extraction (mel-spectrograms, F0, energy, phonetic alignments, self-supervised features from HuBERT/Whisper). The primary research focus is on high-quality, zero-shot, and controllable generation using architectures like diffusion models, flow matching, GANs, autoregressive transformers, and vector-quantized codecs. The repository also includes evaluation metrics (F0, energy, intelligibility, speaker similarity) and visualization tools (SingVisio). While primarily a research codebase, it includes inference scripts and Gradio demos for practical use. External dependencies include PyTorch, HuggingFace Transformers, WeNet, Descript Audio Codec, and various pretrained models.
This repository contains educational labs for MIT's Introduction to Deep Learning course (6.S191). It consists of Jupyter notebooks covering deep learning fundamentals (Lab1: music generation with RNNs), computer vision (Lab2: MNIST classification and facial detection with VAEs), and LLM fine-tuning (Lab3: style adaptation using LoRA). The repository also includes extra labs for reinforcement learning and uncertainty quantification. While structured as educational material, the labs involve substantive data processing, model training, and evaluation pipelines. The mitdeeplearning package provides reusable utilities for training visualization and data handling. The code assumes execution in Google Colab with GPU access and relies on external APIs (Comet ML, OpenRouter) and datasets hosted on third-party services (Dropbox).