This repository provides the data pipeline for the Sekai dataset, a large-scale egocentric video dataset for world exploration and generation research. It contains two main toolchains: dataset_downloading (extracting YouTube URLs from CSV annotations and downloading videos via yt-dlp) and clip_extracting (a multi-step GPU-accelerated pipeline to split raw videos into annotated clips, transcode them to H.265, and remux with audio). The pipeline is designed for the Sekai-Real subset and requires NVIDIA GPUs with NVENC support, CUDA 11.8, and specific Python dependencies. The code is well-structured with clear separation of steps, logging, and multiprocessing support. While tightly coupled to the Sekai dataset and hardware requirements, the utility modules for GPU video decoding/encoding and frame sampling are reusable in other video processing projects.
This repository accompanies the VR-NeRF Eyeful Tower dataset (SIGGRAPH Asia 2023), providing the full software stack for capturing, processing, and reconstructing high-quality HDR images from a robotic multi-camera rig. The hardware consists of 14 Sony A1 cameras mounted on a PAL TIAGo mobile base, controlled via a ROS2 node (EyefulController) that orchestrates drive sequences, dual-exposure bracketing, and pose recording. A Godot-based remote control app (Commandly) offers teleoperation, SLAM mapping, capture management, and status monitoring via ROS# and SSH. The processing pipeline includes RAW debayering, HDR merging (PPNE, Debevec, Max), white balance correction, downscaling, JPEG export, and automated Metashape photogrammetry (alignment, dense cloud, mesh, texture) with COLMAP export for NeRF training. The dataset itself (22 indoor scenes, 4.4 TB) is hosted on AWS S3 and includes HDR EXR images at multiple resolutions, camera calibration, and COLMAP reconstructions. While the primary output is a research dataset, the repository contains substantial application-level code: a camera driver library abstracting the Sony Camera Remote SDK, a ROS2 sensor validation node (IMU, LiDAR), and a full-featured Godot UI. These components are tightly coupled to the specific hardware (Sony A1, PAL TIAGo, Ouster LiDAR, Xsens IMU) and proprietary dependencies (Sony SDK, Metashape, LibRaw). The pipeline scripts are parameterized and can be run individually, but require manual environment setup and specific directory structures. The code is functional and achieves its research goals, but is not designed for general reuse without significant adaptation.