vit-pytorch is a comprehensive PyTorch library implementing dozens of Vision Transformer (ViT) architectures and extensions, along with self-supervised pretraining methods (MAE, DINO, SimMIM, MP3, LeJEPA) and video transformers (ViViT, NaViT-3D, CCT-3D). It is a research-oriented codebase offering a consistent API across models, all accepting standard hyperparameters like image_size, patch_size, depth, and heads. The library covers a wide spectrum of attention mechanisms (cross-attention, shifted windows, channel attention, sparse gating), tokenization strategies (patch merging, region-based, t2t, SPT), and training utilities (recorders, extractors, distillation wrappers). While primarily a collection of model implementations, it also includes action transformers for robotics (VAT, VAAT) and N-dimensional ViTs, demonstrating both breadth and depth of recent transformer research. The code is highly modular, relying on shared building blocks such as Attention, FeedForward, and Transformer classes, with heavy but well-organized use of einops for tensor manipulation. Some variants are experimental and may lack the polish of the core models, but the overall architecture is clean and serves as a valuable reference for implementing and comparing ViT variations. The repository is actively maintained, with recent commits and a large user base, but its sheer size and variety mean that not every model is exhaustively tested—core components are covered by basic forward-pass tests, while many advanced variants rely on the underlying patterns being sound.
This is the TensorFlow Model Garden, a large monorepo of model implementations, training loops, and research code. The dominant character is an application/framework: it provides official TF2 model code under `official/`, a custom training library under `orbit/`, and a large body of research code under `research/` and `community/`. The codebase is not a single deployable service but a collection of training/eval/export scripts, model factories, config systems, and data pipelines. It uses Gin configs, dataclass-based configs, and a registry pattern to compose models, tasks, and data loaders. Tests exist across the repo, and CI workflows are present. The repository is hybrid in that it contains research-oriented experiments and data-processing code, but the core is a software framework for building and training TensorFlow models.
This repository is a comprehensive C++ library and suite of command-line tools for mesh processing research, originally developed by Hugues Hoppe at Microsoft Research. It implements algorithms from several SIGGRAPH papers (1992–2003), including surface reconstruction from unorganized points, mesh optimization, progressive meshes, view-dependent refinement, subdivision surface fitting, spherical parameterization, and gradient-domain video looping. The core library, libHh, provides a rich set of reusable data structures (e.g., fixed-size vectors, D-dimensional grids, half-edge meshes, progressive mesh representations) and algorithms (spatial search, least-squares solvers, multigrid, image filtering, parallel patterns). The tools are designed as stream-oriented filters that read from stdin and write to stdout, enabling Unix-style pipelines for complex processing workflows. An interactive OpenGL viewer (G3dOGL) supports real-time rendering with level-of-detail and geomorph transitions. The build system supports multiple platforms (Windows, Linux, macOS via Cygwin/MinGW) using GNU make and Visual Studio, with optional dependencies on libpng, libjpeg, and ffmpeg for media I/O. The codebase is well-tested with a comprehensive suite of unit and integration tests, and includes numerous demo scripts that reproduce published research results.
This repository is a massive, multi-disciplinary research codebase from Google Research, primarily focused on machine learning, optimization, and computer science experiments. While it contains a few application-like components (e.g., a Next.js frontend for model exploration, a Rust-based 3D splatting renderer), the dominant character is data_research, hosting thousands of independent experimental projects. The codebase serves as a central hub for publishing reproducible research in domains including Reinforcement Learning, Computer Vision (NeRF, 3D), NLP (Transformers, LLMs), and Privacy. It relies heavily on shell scripts for environment setup and experiment orchestration, with a mix of Python, C++, Rust, and Java implementations. The infrastructure is robust for research (e.g., `scann` for search, `lila` for experiment management) but exhibits significant fragmentation and technical debt due to the rapid pace of research and mixed framework usage.
This is a research-grade PyTorch implementation of Danijar Hafner's Dreamer v4 model-based reinforcement learning algorithm. The core artifact is the `dreamer4` package, dominated by a monolithic `dreamer4/dreamer4.py` (~3000 lines) that implements `VideoTokenizer`/`SpaceTimeTokenizer`, `DynamicsWorldModel`, `AudioTokenizer`, and `StateTokenizer`. These classes build causal space-time transformers with separate space/time attention blocks, latent flow-matching diffusion with shortcut consistency, BYOL-style tokenizer pre-training, PoPE attention, and a large zoo of experimental loss terms (latent autoregressive loss, decorrelation, orthogonality, SigReg, self-flow distillation, shortcut loss). `dreamer4/trainers.py` provides training loops (VideoTokenizerTrainer, BehaviorCloneTrainer, DreamTrainer, SimTrainer) and `dreamer4/cli.py` exposes CLI entry points for training and serving the world model. The repository is hybrid: primarily research code, but packaged and tested like a library. It ships a pyproject.toml with pinned dependencies, a parametrized test suite (tests/test_dreamer.py, test_e2e.py, test_env_wrapper.py), a browser-based Snake environment with a Flask-style server (`dreamer4/web_env/`) used for human demonstration collection and replay-buffer inspection, and environment wrappers in `dreamer4/env.py` that record agent experience to folders or memmap replay buffers. Root-level scripts are the actual experiment entry points, targeting standard benchmarks: HalfCheetah (imagination RL), CartPole (dynamics + RL), MovingMNIST (tokenizer + dynamics), and Snake (PPO with dynamics model). Craftsmanship is above typical research code: clean package layout, pytest coverage across many hyperparameter combinations, self-contained utilities (LossNormalizer, tensor pack/unpack helpers, with_seed decorator, memmap replay-buffer helpers), and HuggingFace Accelerate integration for distributed/mixed-precision training. However, the single-file core, 100+ hyperparameter flags on trainers, scattered hard-coded constants, sparse docstrings, and reliance on volatile niche packages (x-transformers, h-net-dynamic-chunking, PoPE_pytorch, adam-atan2-pytorch) make long-term stability and code reuse risky. The repository is recently committed and appears maintained, but it is explicitly beta-status experimental research code, not a production system.
This repository is a research evaluation harness for Visual Place Recognition (VPR) models. It wraps a wide range of state-of-the-art VPR methods behind a common CLI, allowing users to run any supported model on standard VPR datasets (e.g., St Lucia) or a bundled toy dataset. The main entry point is test_dataset.py, which loads database/query image folders, extracts descriptors using the selected model, computes recall@N metrics, and saves logs and optional visualizations. The repo is designed to work with the companion VPR-datasets-downloader to standardize dataset format. Although it is a CLI tool, its primary purpose is research evaluation, so it is classified as data_research.
This repository is the official implementation of the SANA family of diffusion models for high-resolution image and video generation, developed by NVIDIA Research. It is primarily a research codebase, containing training and inference pipelines for multiple model variants: SANA (text-to-image), SANA-1.5, SANA-Sprint (few-step distillation), SANA-Video, SANA-Video 2.0, SANA-Streaming (long video), LongSANA, and SANA-WM (world model for camera-controlled video). The codebase also includes post-training with reinforcement learning (Sol-RL), ControlNet support, and various schedulers (flow matching, DPM-Solver, LCM, self-forcing). While the core purpose is research, the repository also provides deployable Gradio applications, conversion scripts to Hugging Face diffusers format, and integration with ComfyUI and SGLang, giving it a hybrid application character. The code is heavily optimized with custom Triton kernels for efficient attention (LiteMLA, Gated Delta Net) and includes distributed training utilities (FSDP, context parallelism). The repository is actively maintained with frequent commits and supports a wide range of model sizes (600M to 14B parameters).
Depth Anything V2 is a research codebase for monocular depth estimation, providing both relative and metric depth models. It builds on a DINOv2 encoder with a DPT decoder, offering four model scales (Small, Base, Large, Giant). The repository includes inference scripts for images and videos, a Gradio demo, and a subdirectory for fine-tuning on metric depth estimation using Hypersim (indoor) and Virtual KITTI 2 (outdoor) datasets. It also defines the DA-2K evaluation benchmark for relative depth. The code is well-structured with reusable transform utilities and evaluation metrics, but reproducibility is limited due to hardcoded dataset paths and unpinned dependencies. This is primarily a research project with a hybrid nature due to the included demo application.
VOID is a Netflix research codebase for removing objects from videos while also erasing their physical side effects, such as shadows, reflections, and secondary motion caused by the removed object. The core runtime is a two-pass diffusion inpainting system built on CogVideoX: pass 1 performs coarse object removal guided by a quadmask, and pass 2 refines temporal consistency using warped noise and optical-flow-inspired conditioning. The repo is primarily an application/research library rather than a pure data project: it ships a `videox_fun` Python package with model definitions, custom diffusion pipelines, inference and training scripts, a FastAPI/Ray inference API, Gradio UIs, and a Colab notebook, alongside synthetic data-generation code that uses Blender and Kubric.
Omnizart is an automatic music transcription (AMT) library that converts polyphonic audio into symbolic representations including pitched notes, drums, chords, vocal melody, and beat/downbeat. Built as a Python CLI tool with multiple subcommands (music, drum, chord, vocal, vocal-contour, beat, patch-cnn), it provides pre-trained TensorFlow models and supports custom training and feature extraction. The architecture follows a modular pattern with separate app, inference, features, labels, and prediction modules per task, all inheriting from common base classes (BaseTranscription, BaseDatasetLoader). While primarily an application library for end-to-end transcription, it has strong research components: it uses standard MIR datasets (MAESTRO, MusicNet, MedleyDB, etc.), implements state-of-the-art models (U-Net with ASPP, attention-based U-Net, PyramidNet, Chord Transformer), and includes custom feature extraction (CFP, CQT, chroma). The repository also bundles a JavaScript waveform playlist library for documentation demos. The codebase shows active maintenance (recent commits in 2026, CI testing up to Python 3.14) but has technical drift risks due to heavy reliance on TensorFlow 1.x compat layers, external dependencies (madmom, vamp), and known bugs in drum model training.
CARI4D is the official CVPR 2026 research codebase for category-agnostic 4D reconstruction of human-object interaction from monocular video. It is fundamentally a research pipeline rather than a production application: raw RGB-D video (primarily BEHAVE, plus HODome/InterCap and custom clips) is processed through a chain of external foundation models - UniDepth for metric depth, NLF/SAM3D for human pose, FoundationPose for object tracking - then aligned, rendered into H5 training files, refined by a learned temporal transformer (CoCoNet), and finally optimized at test time to output temporally consistent SMPL-H body parameters, object 6D pose, and hand contact labels. The repo contains training, inference, preprocessing, and evaluation entry points, but no API/service layer; the runnable scripts are research pipeline drivers.
The code answers the research question of how to customize large text-to-image diffusion models with minimal data and minimal parameter updates, and it compares against DreamBooth and Textual Inversion. It includes reusable components for few-shot personalization, delta checkpoint handling, lightweight model compression, multi-concept weight composition, and CLIP/DINO evaluation, although the codebase shows research-level roughness: duplicated CompVis/diffusers paths, hardcoded paths, and several known bugs in conversion and composition utilities.
EasyMocap is a multi-view 3D human motion capture toolbox designed for research and prototyping. It provides end-to-end pipelines from video input to 3D body mesh recovery, including camera calibration, 2D keypoint detection (via OpenPose, HRNet, MediaPipe, YOLO), multi-view association and triangulation, SMPL/SMPL-X body model fitting, and neural body rendering. The architecture is modular: a config-driven factory pattern (via YAML and `load_object`) instantiates pipeline stages (detection, triangulation, fitting, visualization), and a command-line entry point `emc` orchestrates the workflow. The repository also contains annotation tools, a real-time 3D visualization server using Open3D, and neural body research code (NeRF-based) for novel view synthesis. Despite its research character, the core pipeline is structured as a reusable application framework with separate packages for estimators, operations, and visualization. However, the codebase exhibits significant technical debt: hardcoded paths, duplicated modules (e.g., two HRNet implementations), incomplete error handling, mixed Chinese/English comments, and stale vendored dependencies (Eigen 3.3.7, pybind11). There are no unit tests, and many scripts rely on external model checkpoints that may not be accessible. The repository is actively maintained with recent commits, but the code quality reflects a fast-moving research project rather than a production-grade system.
This repository contains the reference implementation of AugMix, a data augmentation technique for improving robustness and uncertainty calibration in image classification models. It provides training scripts for CIFAR-10/100 and ImageNet using PyTorch, along with evaluation on their corrupted versions (CIFAR-C, ImageNet-C). The core augmentation logic is implemented in NumPy and PIL, and the training code applies a Jensen-Shannon Divergence consistency loss between augmented and clean images. The codebase is research-oriented, with scripts that are well-structured but heavily tied to the specific experimental setup of the paper. Dependencies are pinned but severely outdated (torch==1.2.0), making reproduction on modern systems challenging without adaptation.
This repository implements multiple state-of-the-art Visual Question Answering (VQA) models from 2018, including bottom-up top-down attention, Bilinear Attention Networks (BAN), inter-intra modality attention, learning to count, and conditioned graph structures. It is built on Cyanogenoid's vqa-counting framework and provides a clean, modular codebase for training and evaluating VQA models on the VQA v2 dataset. The code includes data preprocessing scripts (feature extraction, vocabulary building), a flexible training pipeline with various loss functions and learning rate schedules, and evaluation scripts that compute per-category accuracy. While the code is well-structured for research purposes and contains reusable components like the counting module and fusion layers, it relies on deprecated PyTorch APIs (e.g., Variable), hardcoded paths, and lacks unit tests or environment definitions, making reproducibility challenging. The repository is primarily a research artifact for comparing attention mechanisms and counting strategies in VQA.
This repository contains the research code for the paper 'What's hidden in a randomly weighted neural network?' (arXiv 1911.13299). It implements methods for discovering high-performing subnetworks within randomly initialized neural networks without training the weights, building on the lottery ticket hypothesis. The codebase is organized around PyTorch model definitions (ResNet, custom ConvNets), config-driven experiment management (YAML), and modular trainers for different pruning strategies (default, lottery, sample-based sparsification). The core contribution is a set of custom convolution layers (SubnetConv, SampleSubnetConv, FixedSubnetConv) that learn binary masks over frozen weights via straight-through gradient estimators. The repository supports experiments on CIFAR-10, ImageNet, and MNIST, with extensive configuration files for small-scale and large-scale experiments. While the code is well-structured for research reproducibility, it relies on outdated dependencies (PyTorch 1.3.0, torchvision 0.4.1) and contains hardcoded paths, making it primarily useful as a reference implementation rather than a reusable library.