$ show facebookresearch/EyefulTower
This repository accompanies the VR-NeRF Eyeful Tower dataset (SIGGRAPH Asia 2023), providing the full software stack for capturing, processing, and reconstructing high-quality HDR images from a robotic multi-camera rig. The hardware consists of 14 Sony A1 cameras mounted on a PAL TIAGo mobile base, controlled via a ROS2 node (EyefulController) that orchestrates drive sequences, dual-exposure bracketing, and pose recording. A Godot-based remote control app (Commandly) offers teleoperation, SLAM mapping, capture management, and status monitoring via ROS# and SSH. The processing pipeline includes RAW debayering, HDR merging (PPNE, Debevec, Max), white balance correction, downscaling, JPEG export, and automated Metashape photogrammetry (alignment, dense cloud, mesh, texture) with COLMAP export for NeRF training. The dataset itself (22 indoor scenes, 4.4 TB) is hosted on AWS S3 and includes HDR EXR images at multiple resolutions, camera calibration, and COLMAP reconstructions. While the primary output is a research dataset, the repository contains substantial application-level code: a camera driver library abstracting the Sony Camera Remote SDK, a ROS2 sensor validation node (IMU, LiDAR), and a full-featured Godot UI. These components are tightly coupled to the specific hardware (Sony A1, PAL TIAGo, Ouster LiDAR, Xsens IMU) and proprietary dependencies (Sony SDK, Metashape, LibRaw). The pipeline scripts are parameterized and can be run individually, but require manual environment setup and specific directory structures. The code is functional and achieves its research goals, but is not designed for general reuse without significant adaptation.
The LLM identified 4 files worth highlighting for reuse:
Provides a clean abstract interface for Sony camera control (connect, disconnect, set properties, capture, transfer). The implementation in Camera.cpp handles async property setting and error recovery. This pattern is reusable for any project using the Sony Camera Remote SDK, isolating SDK-specific details behind a well-defined API.
Implements multiple HDR merging algorithms (PPNE, robust PPNE, Debevec, Max) with GPU acceleration via CuPy. Handles RAW debayering, auto-exposure compensation, and output in EXR format. The modular design allows easy addition of new methods and is adaptable to other multi-exposure image sets.
Contains reusable utilities for the data processing pipeline: environment variable validation, subprocess execution with logging, directory creation, and file operations. These functions are parameterized and idempotent, making them suitable for extraction into any ETL or data processing workflow.
Generic ROS2 subscriber implementation for Godot using RosSharp. Handles message deserialization, topic subscription, and callback invocation. This pattern can be reused in any Godot project that needs to communicate with ROS2, abstracting the underlying network protocol.