$ show henry123-boy/SpaTrackerV2
SpaTrackerV2 is a 3D point tracking system that processes video input to estimate camera motion, depth, and track user-selected points in 3D space. It integrates multiple depth estimation models (ZoeDepth, DepthAnything, DepthPro) with a transformer-based tracking architecture, offering both offline (full video) and online (sliding window) processing modes. The system provides a Gradio-based web interface for interactive point selection with SAM assistance and outputs 2D tracking videos, 3D visualizations (HTML), and numerical trajectory data. Evaluation modules support benchmarking against datasets like Sintel and CVO. The codebase demonstrates modular design with reusable components for I/O, geometry, alignment, and visualization, though it relies heavily on specific external models and libraries.
The LLM identified 5 files worth highlighting for reuse:
Provides unified functions for reading/writing images, depth, normals, and segmentation maps with metadata handling via PNG text chunks. Self-contained, minimal dependencies, and applicable to any computer vision project requiring standardized I/O operations.
Contains essential geometric transformations (point cloud operations, coordinate conversions, projection/unprojection, focal length conversions) implemented in PyTorch. Well-structured, dependency-light, and reusable across 3D vision tasks without modification.
Implements robust point cloud/depth alignment using L1/L2 optimization with truncation and iterative reweighted least squares (IRLS). Solves a generic registration problem, requiring only point correspondences, and can be dropped into any alignment pipeline.
Standalone HTML/Three.js visualizer for point clouds, trajectories, and camera frustums with interactive controls (playback, history, settings). Zero dependencies beyond CDN libraries, making it portable for visualizing output from any 3D tracking/reconstruction system.
Implements TAP-Vid 3D metrics for tracking accuracy evaluation, including Jaccard, accuracy, and average precision calculations. Self-contained, follows standard benchmarks, and can be integrated into any tracking project for performance assessment.