E-RayZer is a self-supervised 3D reconstruction system that takes unordered multi-view images as input and outputs estimated camera poses, target-view renders, Gaussian point clouds, and turntable videos. The system uses a transformer-based architecture to estimate camera poses and scene representations, then renders novel views using differentiable Gaussian splatting. It includes a Gradio demo interface for easy inference, along with training and evaluation scripts for the DL3DV dataset. The model is trained without ground truth camera poses or depth information, learning to reconstruct 3D scenes purely from multi-view image consistency.
This is a research codebase for the Geometric Context Transformer (GCT), a streaming monocular 3D reconstruction model accepted at ECCV 2026. The system processes RGB video frames in real-time (targeting ~20 FPS) to predict camera poses, depth maps, and 3D point clouds using a vision transformer with a novel paged KV-cache causal attention mechanism. It includes a full inference pipeline, interactive and offline renderers (based on Viser and Open3D/CUDA voxelization with octree LOD), and a comprehensive benchmark framework supporting multiple datasets (KTH, Oxford Spires, TUM, 7-Scenes, ETH3D, etc.) with trajectory, depth, and point cloud metrics. The codebase is predominantly application-oriented—providing a deployable model and evaluation harness—but incorporates significant data engineering components such as dataset preprocessing (e.g., Oxford Spires fisheye-to-pinhole conversion with CUDA visibility filtering), standardized BSS artifact storage, and reproducible conda-based environment setup. The core model architecture demonstrates advanced engineering practices: two-stream token layouts, sliding-window KV cache eviction with special-token preservation, 3D RoPE, and modular attention backends (FlashInfer, SDPA, custom causal). The benchmark framework is well-architected with pluggable methods and datasets, resumable execution, and static HTML report generation.
SAIL-Recon is a feed-forward Transformer-based system for large-scale Structure-from-Motion (SfM) and visual localization. It processes input images or video to estimate camera poses, depth maps, and generate 3D point clouds without iterative optimization. The architecture combines a Vision Transformer backbone with an alternating attention aggregator for temporal modeling and specialized heads for camera, depth, and track predictions. The system includes both command-line and Gradio-based demo interfaces for end-to-end 3D reconstruction, along with evaluation scripts for standard benchmarks (TUM, 7Scenes, Tank&Temple, etc.). While research-oriented, it is packaged as a usable application with clear entry points and modular components.
SpaTrackerV2 is a 3D point tracking system that processes video input to estimate camera motion, depth, and track user-selected points in 3D space. It integrates multiple depth estimation models (ZoeDepth, DepthAnything, DepthPro) with a transformer-based tracking architecture, offering both offline (full video) and online (sliding window) processing modes. The system provides a Gradio-based web interface for interactive point selection with SAM assistance and outputs 2D tracking videos, 3D visualizations (HTML), and numerical trajectory data. Evaluation modules support benchmarking against datasets like Sintel and CVO. The codebase demonstrates modular design with reusable components for I/O, geometry, alignment, and visualization, though it relies heavily on specific external models and libraries.