PyTorch3D is a high-performance library for 3D deep learning, providing differentiable rendering, 3D geometry operations, and neural implicit scene representations. The core library offers efficient data structures for meshes, point clouds, and volumes, with packed/padded batching for variable-size inputs. It includes a full differentiable mesh renderer (rasterization, shading, blending), point cloud renderer (including the Pulsar CUDA backend), and an implicit rendering pipeline for NeRF-style models. The library also provides a comprehensive set of 3D operations (KNN, ball query, farthest point sampling, cubify, marching cubes, IoU for 3D boxes, point-mesh distances) and loss functions (chamfer distance, mesh laplacian smoothing, normal consistency). The architecture is modular: cameras, lights, materials, rasterizers, and shaders are composable components. The implicitron subpackage extends this with a configurable framework for novel view synthesis, featuring dataset providers (CO3D, ShapeNet, R2N2), ray samplers, raymarchers, view poolers, and evaluation metrics. Research projects (NeRF, implicitron trainer) demonstrate end-to-end training pipelines with Hydra/OmegaConf configuration. While primarily a library, the repository has strong research characteristics: it includes benchmark scripts, tutorials, and reproduction configs for multiple papers. The codebase is well-tested with over 100 test files, including numerical equivalence tests and benchmarks. Custom CUDA/C++ kernels ensure performance-critical operations are optimized. The library is designed for reuse, with a registry pattern for components, pluggable IO, and a flexible transform system.
NoPoSplat is a research application for novel view synthesis and camera pose estimation from unposed image pairs. It predicts 3D Gaussian parameters (means, covariances, harmonics, opacities) in canonical space using a vision transformer backbone (CroCo, ResNet, or DINO) with a DPT head, then renders images via differentiable Gaussian splatting. The system eliminates the need for known camera poses during training/inference by estimating poses implicitly through geometric consistency losses. Built on PyTorch Lightning with Hydra configuration, it supports multi-GPU training and evaluation on datasets like RealEstate10K, ACID, DTU, and ScanNet++. The codebase emphasizes modularity through component registries, composable loss functions, and chainable data preprocessing shims.
E-RayZer is a self-supervised 3D reconstruction system that takes unordered multi-view images as input and outputs estimated camera poses, target-view renders, Gaussian point clouds, and turntable videos. The system uses a transformer-based architecture to estimate camera poses and scene representations, then renders novel views using differentiable Gaussian splatting. It includes a Gradio demo interface for easy inference, along with training and evaluation scripts for the DL3DV dataset. The model is trained without ground truth camera poses or depth information, learning to reconstruct 3D scenes purely from multi-view image consistency.
This repository implements Mobile-GS, a real-time Gaussian splatting approach for novel view synthesis optimized for mobile devices. The system trains 3D Gaussian representations from multi-view images, applies compression techniques (vector quantization, Huffman coding, GPCC) to reduce model size, and renders novel views via differentiable EWA (Elliptical Weighted Average) splatting. It includes training pipelines (train.py, pretrain.py), rendering engine (gaussian_renderer/), scene management, custom CUDA kernels for efficient rasterization, and evaluation metrics. The architecture follows a modular design with separate components for training, rendering, scene handling, and compression, optimized for deployment on mobile platforms with limited computational resources.