This is a research codebase for the Geometric Context Transformer (GCT), a streaming monocular 3D reconstruction model accepted at ECCV 2026. The system processes RGB video frames in real-time (targeting ~20 FPS) to predict camera poses, depth maps, and 3D point clouds using a vision transformer with a novel paged KV-cache causal attention mechanism. It includes a full inference pipeline, interactive and offline renderers (based on Viser and Open3D/CUDA voxelization with octree LOD), and a comprehensive benchmark framework supporting multiple datasets (KTH, Oxford Spires, TUM, 7-Scenes, ETH3D, etc.) with trajectory, depth, and point cloud metrics. The codebase is predominantly application-oriented—providing a deployable model and evaluation harness—but incorporates significant data engineering components such as dataset preprocessing (e.g., Oxford Spires fisheye-to-pinhole conversion with CUDA visibility filtering), standardized BSS artifact storage, and reproducible conda-based environment setup. The core model architecture demonstrates advanced engineering practices: two-stream token layouts, sliding-window KV cache eviction with special-token preservation, 3D RoPE, and modular attention backends (FlashInfer, SDPA, custom causal). The benchmark framework is well-architected with pluggable methods and datasets, resumable execution, and static HTML report generation.
This repository contains the LatentNeeds-Bench benchmark and evaluation framework for proactive AI demand detection, as described in the research paper 'Pask: Toward Intent-Aware Proactive Agents with Long-Term Memory'. The project focuses on creating a benchmark to evaluate whether AI models can detect latent user needs in ongoing conversations without explicit prompting. It includes a comprehensive data processing pipeline that transforms raw speech transcripts into annotated benchmark datasets, an evaluation framework to test various LLMs on this benchmark, and tools to generate research paper tables and figures. The system implements the DD-MM-PAS paradigm (Demand Detection, Memory Modeling, Proactive Agent System) with IntentFlow as the specialized demand detection model.