$ show hku-mars/GS-SDF
GS-SDF is a research prototype from HKU-MARS that fuses LiDAR-augmented 3D Gaussian Splatting with a neural Signed Distance Field (SDF) for photorealistic rendering and high-fidelity surface reconstruction. The system ingests data from multiple sensor modalities (LiDAR, RGB-D, stereo) via a modular data parser hierarchy supporting Replica, FAST-LIVO2, KITTI, COLMAP, Oxford Spires, and other formats. It trains a neural SDF field with eikonal and curvature regularization, then initializes and refines 3D Gaussians guided by the SDF to produce view-dependent color and geometry. The pipeline includes real-time ROS visualization (RVIZ), offline mesh extraction via a custom CUDA marching cubes implementation, and evaluation scripts for image metrics (PSNR, SSIM, LPIPS) and structural metrics (Chamfer distance, F-score). The codebase is pre-release, with many TODOs and commented-out code, but exhibits strong modularity in its data loaders, loss functions, optimizer utilities, and coordinate transformations. While primarily a research codebase, it includes application-like features such as a ROS node with interactive keyboard commands and headless rendering modes.
The LLM identified 6 files worth highlighting for reuse:
Abstract base class for dataset parsers with virtual methods for frame iteration, camera intrinsics, and point cloud loading. Derived parsers (Replica, KITTI, FastLivo, etc.) implement only dataset-specific logic, making it straightforward to add new sensor configurations without modifying the core data loader.
Utility functions for managing Adam optimizer state when parameter tensors change size (pruning, splitting, concatenation). Functions like prune_optimizer, cat_tensors_to_optimizer, and replace_tensors_to_optimizer handle the tricky task of preserving momentum and variance buffers across dynamic tensor operations, a common need in neural rendering pipelines.
Defines RaySamples, DepthSamples, and ColorSamples structs with chainable methods (index, cat, to, clone) for batched ray operations. These lightweight tensor wrappers enable clean composition of SDF and Gaussian sampling without manual tensor slicing, and are reusable in any ray-based neural field project.
Implements coordinate system conversions between OpenCV, Blender, OpenGL, and KITTI conventions. Functions like change_camera_system and change_world_system handle the rotation/axis flips needed when fusing data from different sensors, a common pain point in multi-modal 3D reconstruction.
CUDA kernel for marching cubes surface extraction from a dense density grid. Uses packed tensor accessors for efficient PyTorch integration, builds a vertex index grid, and generates faces via a local edge table. The pattern is reusable for any neural implicit surface extraction and includes a TODO for sparse tensor optimization.
Standalone Python script that converts ROS bag files (with camera info, images, and optional LiDAR scans) into COLMAP-format text files. Includes distortion correction, PSNR-based frame filtering, and blur detection. Can be reused independently to prepare raw ROS data for any COLMAP-based pipeline.