$ show cvg/NoPoSplat
NoPoSplat is a research application for novel view synthesis and camera pose estimation from unposed image pairs. It predicts 3D Gaussian parameters (means, covariances, harmonics, opacities) in canonical space using a vision transformer backbone (CroCo, ResNet, or DINO) with a DPT head, then renders images via differentiable Gaussian splatting. The system eliminates the need for known camera poses during training/inference by estimating poses implicitly through geometric consistency losses. Built on PyTorch Lightning with Hydra configuration, it supports multi-GPU training and evaluation on datasets like RealEstate10K, ACID, DTU, and ScanNet++. The codebase emphasizes modularity through component registries, composable loss functions, and chainable data preprocessing shims.
The LLM identified 4 files worth highlighting for reuse:
Defines a base Loss class and MultiLoss class enabling composable loss functions via operator overloading (e.g., loss = Loss1() + 0.1*Loss2()). Isolated, dependency-light, and reusable in any PyTorch project for flexible loss weighting.
Implements a callable shim for image normalization. Part of a chainable preprocessing pipeline where each shim (normalization, augmentation, cropping) independently modifies a sample. Highly reusable for image-based datasets due to minimal dependencies and single-responsibility design.
Implements a registry pattern for backbone networks (ResNet, DINO, CroCo) allowing dynamic instantiation by name. Decouples architecture from backbone choice, enabling easy extension with new backbones through simple registration.
Provides utilities to load and adapt pretrained weights, handling architecture mismatches (e.g., layer interpolation/repetition). Self-contained, PyTorch-only, and useful for transfer learning in vision models without modifying core training logic.