PyTorch3D is a high-performance library for 3D deep learning, providing differentiable rendering, 3D geometry operations, and neural implicit scene representations. The core library offers efficient data structures for meshes, point clouds, and volumes, with packed/padded batching for variable-size inputs. It includes a full differentiable mesh renderer (rasterization, shading, blending), point cloud renderer (including the Pulsar CUDA backend), and an implicit rendering pipeline for NeRF-style models. The library also provides a comprehensive set of 3D operations (KNN, ball query, farthest point sampling, cubify, marching cubes, IoU for 3D boxes, point-mesh distances) and loss functions (chamfer distance, mesh laplacian smoothing, normal consistency). The architecture is modular: cameras, lights, materials, rasterizers, and shaders are composable components. The implicitron subpackage extends this with a configurable framework for novel view synthesis, featuring dataset providers (CO3D, ShapeNet, R2N2), ray samplers, raymarchers, view poolers, and evaluation metrics. Research projects (NeRF, implicitron trainer) demonstrate end-to-end training pipelines with Hydra/OmegaConf configuration. While primarily a library, the repository has strong research characteristics: it includes benchmark scripts, tutorials, and reproduction configs for multiple papers. The codebase is well-tested with over 100 test files, including numerical equivalence tests and benchmarks. Custom CUDA/C++ kernels ensure performance-critical operations are optimized. The library is designed for reuse, with a registry pattern for components, pluggable IO, and a flexible transform system.
NoPoSplat is a research application for novel view synthesis and camera pose estimation from unposed image pairs. It predicts 3D Gaussian parameters (means, covariances, harmonics, opacities) in canonical space using a vision transformer backbone (CroCo, ResNet, or DINO) with a DPT head, then renders images via differentiable Gaussian splatting. The system eliminates the need for known camera poses during training/inference by estimating poses implicitly through geometric consistency losses. Built on PyTorch Lightning with Hydra configuration, it supports multi-GPU training and evaluation on datasets like RealEstate10K, ACID, DTU, and ScanNet++. The codebase emphasizes modularity through component registries, composable loss functions, and chainable data preprocessing shims.
This repository implements Mobile-GS, a real-time Gaussian splatting approach for novel view synthesis optimized for mobile devices. The system trains 3D Gaussian representations from multi-view images, applies compression techniques (vector quantization, Huffman coding, GPCC) to reduce model size, and renders novel views via differentiable EWA (Elliptical Weighted Average) splatting. It includes training pipelines (train.py, pretrain.py), rendering engine (gaussian_renderer/), scene management, custom CUDA kernels for efficient rasterization, and evaluation metrics. The architecture follows a modular design with separate components for training, rendering, scene handling, and compression, optimized for deployment on mobile platforms with limited computational resources.
E-RayZer is a self-supervised 3D reconstruction / spatial visual pre-training research codebase. In practice, it takes unordered multi-view RGB images, estimates camera poses via a transformer-based PoseEstimator, constructs a per-pixel 3D Gaussian scene representation, and renders novel views via differentiable Gaussian splatting (gsplat). The repository includes a Gradio demo, a reusable inference engine, training/evaluation drivers for the DL3DV dataset, and benchmark split tooling for ScanNet++ and BlendedMVS. The code is organized into model modules (transformer encoder/decoder, pose estimator, Gaussian upsampler/renderer), utility layers (camera geometry, loss functions including VGG19 perceptual loss), and data-preparation scripts (Hugging Face downloader, OpenCV list conversion, covisibility/DINOv2 statistics, manifest checking). Training uses curriculum view selection, mixed precision, DDP, gradient accumulation, and configurable loss composition; evaluation reports PSNR and pose RPA metrics. Although the dominant character is research/data engineering, the repo is hybrid: it ships application-style entry points (gradio_app.py, app_core/engine.py) as well as data pipelines and model training scripts. The public training path is limited to DL3DV; the ScanNet++ and BlendedMVS benchmark scripts are historical protocol references that rely on internal S3 tooling. Reproducibility is moderate: DL3DV setup is documented and scripted, but the custom CUDA renderutils plugin, third-party gsplat submodule, external VGG weights, and curated download URLs make the environment fragile. Model weights are separately licensed under Adobe's non-commercial research license, while code is MIT.