This is a research-grade PyTorch implementation of Danijar Hafner's Dreamer v4 model-based reinforcement learning algorithm. The core artifact is the `dreamer4` package, dominated by a monolithic `dreamer4/dreamer4.py` (~3000 lines) that implements `VideoTokenizer`/`SpaceTimeTokenizer`, `DynamicsWorldModel`, `AudioTokenizer`, and `StateTokenizer`. These classes build causal space-time transformers with separate space/time attention blocks, latent flow-matching diffusion with shortcut consistency, BYOL-style tokenizer pre-training, PoPE attention, and a large zoo of experimental loss terms (latent autoregressive loss, decorrelation, orthogonality, SigReg, self-flow distillation, shortcut loss). `dreamer4/trainers.py` provides training loops (VideoTokenizerTrainer, BehaviorCloneTrainer, DreamTrainer, SimTrainer) and `dreamer4/cli.py` exposes CLI entry points for training and serving the world model. The repository is hybrid: primarily research code, but packaged and tested like a library. It ships a pyproject.toml with pinned dependencies, a parametrized test suite (tests/test_dreamer.py, test_e2e.py, test_env_wrapper.py), a browser-based Snake environment with a Flask-style server (`dreamer4/web_env/`) used for human demonstration collection and replay-buffer inspection, and environment wrappers in `dreamer4/env.py` that record agent experience to folders or memmap replay buffers. Root-level scripts are the actual experiment entry points, targeting standard benchmarks: HalfCheetah (imagination RL), CartPole (dynamics + RL), MovingMNIST (tokenizer + dynamics), and Snake (PPO with dynamics model). Craftsmanship is above typical research code: clean package layout, pytest coverage across many hyperparameter combinations, self-contained utilities (LossNormalizer, tensor pack/unpack helpers, with_seed decorator, memmap replay-buffer helpers), and HuggingFace Accelerate integration for distributed/mixed-precision training. However, the single-file core, 100+ hyperparameter flags on trainers, scattered hard-coded constants, sparse docstrings, and reliance on volatile niche packages (x-transformers, h-net-dynamic-chunking, PoPE_pytorch, adam-atan2-pytorch) make long-term stability and code reuse risky. The repository is recently committed and appears maintained, but it is explicitly beta-status experimental research code, not a production system.
This repository is a research evaluation harness for Visual Place Recognition (VPR) models. It wraps a wide range of state-of-the-art VPR methods behind a common CLI, allowing users to run any supported model on standard VPR datasets (e.g., St Lucia) or a bundled toy dataset. The main entry point is test_dataset.py, which loads database/query image folders, extracts descriptors using the selected model, computes recall@N metrics, and saves logs and optional visualizations. The repo is designed to work with the companion VPR-datasets-downloader to standardize dataset format. Although it is a CLI tool, its primary purpose is research evaluation, so it is classified as data_research.
GrowSP is an unsupervised semantic segmentation framework for 3D point clouds that learns from growing superpoints. The system processes raw point cloud data through preparation pipelines, generates initial superpoints using voxelization and region growing, then trains a Feature Pyramid Network (Res16FPN18) to learn discriminative features. During training, superpoints are progressively merged (growing superpoints strategy) to create hierarchical representations, with pseudo-labels generated via clustering of superpoint features. The framework includes specialized dataset classes for S3DIS, ScanNet, and SemanticKITTI, evaluation scripts using Hungarian matching for unsupervised performance measurement, and utility libraries for 3D augmentations and PLY file handling. The architecture follows a modular design with clear separation between data preparation, dataset handling, model definitions, and training/evaluation logic.
Stable Diffusion Web UI is a Gradio-based web application that provides a comprehensive interface for generating and manipulating images using diffusion models. It supports txt2img, img2img, inpainting, outpainting, and various post-processing operations including face restoration and upscaling. The application implements a plugin architecture through extensions, supports multiple model architectures (SD 1.x, SD 2.x, SDXL, SD3), and provides a REST API for programmatic access. The codebase is a monolithic application with tightly coupled modules handling model loading, image processing, UI rendering, and script execution. The architecture centers around a main webui.py entry point that initializes the application, sets up Gradio interfaces, and manages the event loop. The modules directory contains core functionality including sd_models.py for model management, processing.py for the main generation pipeline, and various hijack modules that monkey-patch diffusion model components to add custom behavior like attention optimization and CLIP modification. The extensions-builtin directory contains first-party extensions for LoRA, hypernetworks, and various post-processing operations. Key technical patterns include extensive use of monkey-patching and function hijacking to modify model behavior without changing upstream code, a callback system for script hooks, and a queue-based generation system with progress tracking. The codebase demonstrates sophisticated handling of GPU memory management, model optimization techniques, and cross-platform compatibility. However, the code exhibits significant technical debt with large monolithic files, tight coupling between modules, and complex conditional logic handling multiple model versions and configurations. The repository includes comprehensive test coverage for API endpoints, model utilities, and image processing functions. Dependencies are well-pinned in requirements_versions.txt, though some versions are notably behind current releases (Gradio 3.41.2, Transformers 4.30.2). The codebase shows active maintenance with recent commits and a detailed changelog tracking features and bug fixes across multiple versions.
Made With ML is an educational data science project that implements an end-to-end ML pipeline for classifying GitHub repositories into categories (NLP, computer vision, MLOps, other) based on their titles and descriptions. The core data lineage flows from raw CSV datasets (dataset.csv, holdout.csv) through text cleaning, tokenization via SciBERT, and stratified splitting into train/val/test sets, followed by distributed training with PyTorch and Ray Train, hyperparameter tuning via HyperOpt, evaluation with slice-based metrics using Snorkel, and serving through a FastAPI/Ray Serve deployment. The project demonstrates MLOps patterns including MLflow experiment tracking, data validation with Great Expectations, behavioral model testing, and CI/CD via GitHub Actions with Anyscale for cloud execution. The codebase is well-structured with clear separation of concerns across data.py, train.py, tune.py, evaluate.py, predict.py, serve.py, models.py, and utils.py. It includes comprehensive tests at the code, data, and model levels. However, the project shows signs of staleness with pinned dependencies from 2023 (Ray 2.7.0, transformers 4.28.1, torch 2.0.0) that have known security vulnerabilities and compatibility issues with current Python versions. The data pipeline components—particularly the CustomPreprocessor, stratified splitting, and text cleaning—are genuinely reusable patterns. The evaluation framework with slice-based metrics and behavioral tests (invariance, directional expectations, minimum functionality) represents solid ML engineering practice. The deployment configuration is tightly coupled to Anyscale's proprietary platform, limiting portability.
This repository is the official implementation of the SANA family of diffusion models for high-resolution image and video generation, developed by NVIDIA Research. It is primarily a research codebase, containing training and inference pipelines for multiple model variants: SANA (text-to-image), SANA-1.5, SANA-Sprint (few-step distillation), SANA-Video, SANA-Video 2.0, SANA-Streaming (long video), LongSANA, and SANA-WM (world model for camera-controlled video). The codebase also includes post-training with reinforcement learning (Sol-RL), ControlNet support, and various schedulers (flow matching, DPM-Solver, LCM, self-forcing). While the core purpose is research, the repository also provides deployable Gradio applications, conversion scripts to Hugging Face diffusers format, and integration with ComfyUI and SGLang, giving it a hybrid application character. The code is heavily optimized with custom Triton kernels for efficient attention (LiteMLA, Gated Delta Net) and includes distributed training utilities (FSDP, context parallelism). The repository is actively maintained with frequent commits and supports a wide range of model sizes (600M to 14B parameters).
This repository provides the official PyTorch implementation of MAR (Masked Autoregressive with Diffusion Loss), a research method for class-conditional image generation on ImageNet 256x256. The core contribution is an autoregressive model that operates on continuous-valued latent tokens (via a pre-trained VAE) and uses a diffusion loss instead of vector quantization. The codebase includes model definitions (MAR encoder-decoder transformer, DiffLoss MLP), training and evaluation scripts with PyTorch DDP, a VAE latent caching pipeline, and pre-trained checkpoints for base/large/huge variants. While primarily a research prototype, it contains reusable components such as the DiffLoss module, diffusion utilities, and a clean data caching pattern. The Gradio app and Colab notebook make the model accessible for interactive use.
TorchFort is a library that bridges PyTorch with Fortran and C/C++ programs, primarily targeting HPC/scientific computing applications. It provides a C API that allows users to load PyTorch models (via TorchScript or built-in MLP), train them, and perform inference directly on user-provided arrays without data copying. The library supports supervised learning as well as reinforcement learning (RL) with several state-of-the-art algorithms: DDPG, TD3, SAC, and PPO. It includes comprehensive RL infrastructure such as replay buffers, rollout buffers, online running normalizers, and distributed training via MPI. Integration with Weights & Biases is available for experiment tracking. Architecturally, TorchFort consists of a thin C API layer, a set of C++ implementations using LibTorch, and Fortran bindings generated via ISO_C_BINDING. The core C++ code is organized into modules for model management, training, distributed communication, CUDA utilities, and RL subsystems. Configuration is externalized via YAML files, and the library is designed to be used as a static or shared library linked into user applications. Notable components include a prioritized replay buffer with sum-tree implementation (replay_buffer.h), an online running normalizer using Welford's algorithm with MPI synchronization (running_normalizer.h), and a tensor list abstraction for handling multiple typed arrays (tensor_list.h). The codebase demonstrates good engineering practices with extensive test coverage, error handling via return codes, and support for both CPU and GPU execution. While primarily an application/library for integrating PyTorch into HPC codes, the repository also contains research-quality RL algorithm implementations and data preprocessing components, giving it a hybrid character.
VoxCPM is a tokenizer-free, multilingual text-to-speech (TTS) system developed by OpenBMB. It supports voice cloning, voice design, and controllable speech generation across 30 languages and 9 Chinese dialects, outputting 48kHz audio. The architecture is a diffusion-autoregressive model built on MiniCPM-4, comprising AudioVAE, Local Encoder, Text-Semantic LM, Residual Acoustic LM, and Local DiT. The repository provides multiple interfaces: a Gradio web application (app.py) for interactive use, a command-line interface (voxcpm.cli) for batch processing, a fine-tuning web UI (lora_ft_webui.py) for LoRA and full-parameter training, and Python APIs for integration. It also includes utilities for audio enhancement (ZipEnhancer), timestamp alignment, and text normalization. While primarily an application, it contains research-grade model definitions and training pipelines, making it a hybrid project with a strong application focus.
Depth Anything V2 is a research codebase for monocular depth estimation, providing both relative and metric depth models. It builds on a DINOv2 encoder with a DPT decoder, offering four model scales (Small, Base, Large, Giant). The repository includes inference scripts for images and videos, a Gradio demo, and a subdirectory for fine-tuning on metric depth estimation using Hypersim (indoor) and Virtual KITTI 2 (outdoor) datasets. It also defines the DA-2K evaluation benchmark for relative depth. The code is well-structured with reusable transform utilities and evaluation metrics, but reproducibility is limited due to hardcoded dataset paths and unpinned dependencies. This is primarily a research project with a hybrid nature due to the included demo application.
This repository implements a two-stage generative image model: Representation Autoencoders (RAE) followed by Diffusion Transformers (DiT^DH). Stage 1 trains a ViT decoder to reconstruct images from frozen pretrained encoders (DINOv2, MAE, SigLIP), with optional GAN and LPIPS losses. Stage 2 trains a class-conditional diffusion transformer on the resulting latent space, supporting both classifier-free guidance and autoguidance. The codebase is config-driven (OmegaConf), PyTorch DDP, and provides training, distributed sampling, FID evaluation, and latent statistics computation. It is a research codebase with strong engineering quality, modular design (transport, discriminator, encoders, decoders), and reproducibility through pinned dependencies and checkpointing.
This repository is a curated collection of Jupyter notebooks implementing core components of modern LLM architectures, intended for interview preparation. It covers attention mechanisms (MHA, GQA, MLA, KV-cache), normalization layers (LayerNorm, RMSNorm), functional components (RoPE, SwiGLU, LoRA, BPE, activation functions, cross-entropy, InfoNCE, sampling, quantization, SFT loss, tool calling parsing), and reinforcement learning losses (DPO, GRPO, DAPO, GSPO, KL divergence). Each notebook is a self-contained, minimal implementation of a specific algorithm, typically using PyTorch. The code is clean, concise, and well-commented, reflecting good engineering practices for educational purposes. While the repository is not a production application and lacks data pipelines, model training scripts, or continuous integration, it serves as a high-quality reference for understanding and reproducing the mathematical foundations of transformer-based language models. The hybrid nature lies in its focus on research-oriented ML components, but its structure is a pure code library rather than a data-driven research project.
SymTorch is a Python library for deep learning interpretability via symbolic regression. It wraps PyTorch neural network components (layers, submodules) or arbitrary callable functions (e.g., LLM inference, scikit-learn models) and uses the PySR symbolic regression engine (Julia-backed) to discover human-readable mathematical equations that approximate the wrapped component's input-output behavior. The library provides a unified SymbolicModel class that can operate at the layer level (using forward hooks to capture intermediate activations) or in a model-agnostic mode for any black-box function. It also supports local interpretability via SLIME (synthetic neighborhood sampling with weighted regression) and progressive dimensionality reduction through configurable pruning schedules (exponential, linear, cosine decay). The primary workflow involves wrapping a model, optionally training with pruning, calling distill() to run symbolic regression, inspecting discovered equations via show_symbolic_expression(), and optionally switching the forward pass to use the symbolic equation instead of the original block via switch_to_symbolic(). The architecture is monolithic: the SymbolicModel class (~2300 lines) encapsulates all functionality including caching, PySR parameter management, forward hook handling, SLIME sampling, pruning schedules, and serialization via dill. While this centralization simplifies the public API, it creates a single point of maintenance and limits modular reuse. The library relies on several heavy external dependencies: PySR (which requires Julia), PyTorch, SymPy, and dill. Documentation includes comprehensive API reference, Jupyter notebook demos (PINNs, GNNs, LLM arithmetic probing), and case studies demonstrating symbolic extraction from physics-informed neural networks and small language models. The codebase demonstrates solid engineering with good docstrings, extensive caching to avoid redundant computations, and thoughtful design decisions like configurable pruning schedules and SLIME parameters. However, there are signs of academic development: future-dated version numbers (1.1.1 docs vs 1.0.1.post1 setup.py), local path references in demos, and incomplete decomposition (TODO comments about breaking up the class). The testing suite exists but heavily mocks PySR, which may miss integration issues. Overall, SymTorch is a functional, research-oriented interpretability toolkit with clear value for extracting symbolic equations from trained models, but its monolithic structure and external dependencies limit its reuse as a standalone component.
MockingBird is a Chinese voice cloning system implementing the SV2TTS (Speaker Verification to Text-to-Speech) framework. It provides a complete pipeline for few-shot voice cloning: a speaker encoder extracts d-vector embeddings from reference audio, a synthesizer (Tacotron with Global Style Tokens or VITS) converts text to mel-spectrograms conditioned on the speaker embedding, and a vocoder (HiFi-GAN, Fre-GAN, or WaveRNN) generates waveforms. It also includes a PPG-based voice conversion path using a Conformer-based PPG extractor and a PPG2Mel converter. The system supports multiple Chinese datasets (aidatatang_200zh, magicdata, aishell3, data_aishell) and provides web UIs (Streamlit/FastAPI), a PyQt5 toolbox, and CLI tools for training and inference. The codebase is a fork of Real-Time-Voice-Cloning adapted for Mandarin with significant additions including VITS synthesizer, multiple vocoder options, emotion extraction via wav2vec2, and a modern web API layer. While the original author has moved to a commercial product (noiz.ai), the repository shows recent commits (2026) and maintains a large community (36k stars).
Amphion is a comprehensive open-source research toolkit for audio, music, and speech generation, developed by OpenMMLab. It covers multiple generative tasks including text-to-speech (TTS), singing voice conversion (SVC), voice conversion (VC), text-to-audio (TTA), neural audio codecs, and vocoders. The codebase is organized around config-driven experiments (JSON + shell scripts) with modular base classes for trainers, datasets, and inference. It ingests a wide variety of public datasets (LJSpeech, LibriTTS, VCTK, M4Singer, Emilia, etc.) and provides extensive preprocessing pipelines for feature extraction (mel-spectrograms, F0, energy, phonetic alignments, self-supervised features from HuBERT/Whisper). The primary research focus is on high-quality, zero-shot, and controllable generation using architectures like diffusion models, flow matching, GANs, autoregressive transformers, and vector-quantized codecs. The repository also includes evaluation metrics (F0, energy, intelligibility, speaker similarity) and visualization tools (SingVisio). While primarily a research codebase, it includes inference scripts and Gradio demos for practical use. External dependencies include PyTorch, HuggingFace Transformers, WeNet, Descript Audio Codec, and various pretrained models.
MiniMind-O is a research codebase for training and evaluating a small-scale (0.1B parameter) omni-modal language model that processes text, audio, and image inputs and generates text or streaming speech. The architecture uses a Thinker–Talker dual-path design with frozen external encoders (SenseVoice, SigLIP2, Mimi codec, CAM++) and trainable projectors, plus a Multi-Token Prediction (MTP) head for audio codec tokens. The repository includes a complete training pipeline (SFT with DDP, mixed precision, checkpointing), a comprehensive evaluation CLI, a Gradio web demo, and a real-time Flask WebSocket demo with VAD and ASR for near-duplex interaction. Data is stored in parquet format with pre-encoded audio codes, images, and speaker embeddings. The project is primarily research-oriented, aiming to provide a transparent, reproducible baseline for omni-modal models, but also includes application-like components for interactive demos.
This repository is an educational tutorial series that teaches how to build a nanoGPT-like transformer language model from scratch using PyTorch. It consists of over 30 Jupyter notebooks (with both TODO exercises and answer keys) designed for Google Colab, covering topics from token embeddings and positional encodings to multi-head attention, transformer blocks, training loops, scaling experiments, and instruction tuning. The code is modular, with separate classes for each component (TokenEmbedding, PositionEmbedding, AttentionHead, MultiHeadAttention, FeedForward, TransformerBlock, VocabularyLogits, nanoGPT, DataLoader, Trainer, ModelConfig), making it a valuable learning resource. Training is demonstrated on the Shakespeare dataset and later on a sample of FineWeb-Edu, with experiments on model size, learning rate, and tokenization. While the repository is not a production data pipeline or deployable application, it exhibits strong data/research characteristics through its focus on model training, scaling laws, and loss analysis. The notebooks are well-structured for self-paced learning but rely on Colab-specific workflows and hardcoded data URLs, limiting portability.
SimScale is a research codebase for data-driven, non-reactive autonomous vehicle simulation and benchmarking, developed by OpenDriveLab. It provides a comprehensive pipeline for training and evaluating end-to-end driving policies on the nuPlan/OpenScene dataset, with support for synthetic data augmentation (SimScale) and multiple agent architectures including Transfuser, DiffusionDrive, and GTRS-Dense. The repository implements a Plan-Do-Measure (PDM) closed-loop scoring framework, metric caching, and a Hydra-based configuration system for flexible experiment management. While primarily a research platform, it includes production-like training scripts, evaluation pipelines, and submission tools for the NAVSIM v2 challenge. The codebase is heavily coupled to the nuPlan ecosystem and requires large-scale data downloads (>1TB), but offers reusable components such as abstract agent interfaces, feature/target builders, and batch IDM policies. It is actively maintained with recent commits and a growing community of extensions.
OpenHGNN is a comprehensive research framework for Heterogeneous Graph Neural Networks (HGNNs) built on PyTorch and DGL. The repository implements 83+ graph neural network models across 17+ tasks including node classification, link prediction, recommendation, knowledge graph completion, and temporal graph learning. It serves as a benchmarking platform for academic research, providing standardized implementations of state-of-the-art architectures like HGT, MAGNN, CompGCN, and HERO. The framework uses a registration pattern for dynamic model/trainer discovery and supports distributed training, hyperparameter optimization via Optuna, and GraphBolt integration for large-scale graphs. While it has CLI entry points and setup.py, its primary purpose is research reproducibility rather than production deployment. The codebase demonstrates strong architectural patterns with BaseDataset, BaseTask, and BaseFlow abstractions that enforce consistent interfaces across 50+ model implementations. However, significant technical debt exists including version mismatches (setup.py declares 0.7.0 while README claims 0.9.0), hardcoded dataset paths, mixed DGL/PyG usage, and monolithic configuration files. The repository contains extensive training artifacts in output/ directories and documentation drift between README, setup.py, and Sphinx docs. Despite these issues, it remains a valuable resource for graph learning researchers seeking reproducible implementations of heterogeneous GNN architectures.
Scal3R is a research codebase for CVPR 2026 Highlight paper on large-scale 3D reconstruction from unordered images using test-time training. The core contribution is a novel bidirectional attention mechanism with fast weight updates (TTT layers) that adapts during inference to improve reconstruction quality for large scenes. The repository provides an end-to-end inference pipeline: given a folder of images, it estimates camera poses, depth maps, and produces a colored point cloud. The architecture builds on a DINOv2 backbone and VGGT-based aggregator with custom attention heads, loop detection via SALAD, and pose graph optimization with submap management. The codebase is well-structured with separate modules for data loading, model definition, inference orchestration, and visualization tools. It includes fallback implementations for key components (e.g., Python-only loop solver, simple correlation-based loop detector) to improve portability, though GPU and CUDA are assumed. The repository is primarily a research artifact with clean utilities that could be extracted for other 3D vision projects. Evaluation code and dataset scripts are explicitly marked as not yet released, limiting reproducibility until those are provided.