$ show cedarconnor/SPAG4d
SPAG4D is a production-grade tool that converts 360° equirectangular panoramic photos into explorable 3D Gaussian Splat (3DGS) scenes. It integrates multiple depth estimation backends (DA360, DAP, PaGeR, SHARP, UniSHARP) to generate depth maps from a single panorama, then projects the points into standard 3DGS PLY format. The system provides a CLI, a FastAPI web server with a JavaScript frontend, and a Python API for programmatic use. Beyond the core conversion, it includes optional refinement pipelines to fill holes and improve scene quality: OmniRoam (WSL2-based video generation and distillation), ArtiFixer3D (Docker-based diffusion refinement), and a lighter geometric refinement pipeline using depth alignment and voxel aggregation. The architecture is modular with pluggable generators, comprehensive configuration dataclasses, and provenance tracking. While the tool is primarily an application, it leans heavily on research-grade depth models and refinement techniques, giving it a hybrid character. The codebase is well-structured with clear separation of concerns, reusable utilities (spherical projection, depth blending, scene filtering, geometric alignment), and extensive tests. It is actively maintained with recent commits, though it carries complexity from multiple backends and platform-specific dependencies (WSL2, Docker, CUDA).
The LLM identified 6 files worth highlighting for reuse:
Provides a clean `SphericalGrid` dataclass and `create_spherical_grid()` function that encapsulates the geometry of an equirectangular panorama (width, height, azimuth, elevation, directions). This is reusable for any project that needs to sample or project ERP coordinates, independent of the rest of the pipeline.
Implements `DepthBlender` with three blending strategies (Laplacian pyramid, Poisson, feathered) for merging depth maps from multiple views. The class is self-contained, well-parameterized, and demonstrates non-obvious image processing skill. Can be extracted for any multi-view depth fusion task.
Contains functions for sky detection (depth/gradient-based), pole thinning, grazing-angle pruning, and sparse region removal. Each filter is an independent, well-documented function that operates on numpy arrays. Useful for any point cloud cleaning pipeline.
Implements IRLS-based scale/shift alignment with Huber loss for aligning per-frame depth maps to a reference. The algorithm is robust to outliers and uses a closed-form solution for the weighted least-squares step. Reusable for any depth alignment problem across frames or sensors.
Implements a cross-frame consistency gate using KDTree to filter out Gaussian candidates that are not supported by neighboring views. This is a clean, self-contained pattern for rejecting spurious points in multi-view reconstruction. The logic is modular and can be repurposed for any point cloud validation task.
Provides provenance tagging for Gaussians using constant tags (original, densified, OmniRoam, ArtiFixer3D) and helper functions to tag ranges and create masks. This is a simple but effective pattern for tracking the origin of points in a multi-stage pipeline. Reusable for any system that needs to attribute contributions to different processing stages.