$ show foxglove/mcap
The foxglove/mcap repository implements the MCAP (Message Communication Archive Protocol), a binary serialization format designed for high-performance robotics data logging. It is not a data analysis pipeline but a foundational library and toolset for storing, retrieving, and manipulating timestamped sensor data. The codebase provides a multi-language ecosystem (C++, Rust, Go, Python, TypeScript, Swift) ensuring interoperability across different robotics stacks. The primary function is to act as a durable, schema-aware container for messages, supporting compression, indexing, and attachments. Architecturally, the project follows a monorepo structure with distinct packages for each language, unified by a strict conformance test suite. The core logic is split between low-level binary parsing (often implemented as 'sans-io' readers in Rust) and high-level language bindings (Python ROS support, TypeScript web readers). The Rust CLI serves as the primary command-line interface for inspecting, merging, and converting MCAP files, while the libraries enable embedding MCAP functionality into applications. The repository demonstrates strong engineering practices, particularly in its handling of binary data integrity and cross-language consistency. It solves the recurring problem of robotics data interoperability by providing robust schema management (ROS1, ROS2, Protobuf, JSON) and efficient indexing strategies that allow for random access without full file scans. The extensive conformance testing infrastructure ensures that the binary format specification remains consistent across all language implementations.
The LLM identified 4 files worth highlighting for reuse:
Implements a sans-io architecture for MCAP record parsing, decoupling binary state machine logic from I/O operations. This pattern allows the same parsing logic to be reused across synchronous, asynchronous, and memory-mapped contexts. It demonstrates non-obvious skill in managing complex binary record states (Header, Chunk, Message, Footer) without blocking I/O, ensuring high performance and flexibility.
Provides a generic rewrite engine for merging multiple MCAP files into a single output. It handles complex logic such as schema and channel ID deduplication, coalescing, and maintaining index integrity across inputs. This component is reusable for any binary format manipulation task requiring stream-based transformation and metadata reconciliation.
Solves the recurring problem of dynamic ROS2 message deserialization by generating parsers from schema definitions at runtime. It handles complex type coercion and array handling for CDR-encoded messages. This component is reusable for any system requiring schema-driven binary message decoding without pre-compiled code.
Defines an abstract base class for conformance test execution, standardizing how different language implementations validate against the MCAP specification. It enables the repository to maintain binary format consistency across C++, Rust, Python, and other languages by running identical test variants against each implementation.