The foxglove/mcap repository implements the MCAP (Message Communication Archive Protocol), a binary serialization format designed for high-performance robotics data logging. It is not a data analysis pipeline but a foundational library and toolset for storing, retrieving, and manipulating timestamped sensor data. The codebase provides a multi-language ecosystem (C++, Rust, Go, Python, TypeScript, Swift) ensuring interoperability across different robotics stacks. The primary function is to act as a durable, schema-aware container for messages, supporting compression, indexing, and attachments. Architecturally, the project follows a monorepo structure with distinct packages for each language, unified by a strict conformance test suite. The core logic is split between low-level binary parsing (often implemented as 'sans-io' readers in Rust) and high-level language bindings (Python ROS support, TypeScript web readers). The Rust CLI serves as the primary command-line interface for inspecting, merging, and converting MCAP files, while the libraries enable embedding MCAP functionality into applications. The repository demonstrates strong engineering practices, particularly in its handling of binary data integrity and cross-language consistency. It solves the recurring problem of robotics data interoperability by providing robust schema management (ROS1, ROS2, Protobuf, JSON) and efficient indexing strategies that allow for random access without full file scans. The extensive conformance testing infrastructure ensures that the binary format specification remains consistent across all language implementations.
This is the `tinylib/msgp` repository, a high-performance MessagePack serialization library for Go with an integrated code generation tool. The actual system consists of two layers: a runtime library in the `msgp/` package providing buffered readers/writers, zero-copy byte slice operations, and interfaces like `Marshaler`/`Unmarshaler`; and a code generator (`main.go`, `gen/`, `parse/`) that produces optimized, reflection-free serialization methods from Go structs and directives. The architecture is a combination of a CLI tool (for `//go:generate` workflows) and a library that the generated code depends on. Key entry points include the `Run()` function in `main.go` for code generation, and the `Reader`/`Writer` types for runtime I/O. External dependencies are minimal: `philhofer/fwd` for buffered I/O and `golang.org/x/tools` for code formatting. The library supports advanced features like shims, type replacement, omitempty/omitzero, limits, generics, and set types. The codebase is well-structured, with extensive tests and benchmarks, and demonstrates sophisticated engineering in code generation and serialization performance.
zpp_bits is a modern C++20 header-only binary serialization library that provides high-performance, zero-overhead serialization and deserialization of C++ objects. It supports multiple wire formats including a custom compact binary format and an optional protobuf-compatible protocol (zpp::bits::pb) for cross-language interoperability. The library also includes a built-in RPC framework (zpp::bits::rpc) that can generate client/server stubs from function bindings, and compile-time utilities for hashing (SHA-1, SHA-256) and byte manipulation. The architecture is template-heavy, leveraging C++20 concepts, constexpr evaluation, and structured bindings to automatically serialize aggregates without explicit reflection. Error handling is based on std::errc with optional coroutine support via zpp_throwing. The library is self-contained in a single header file (zpp_bits.h) with no external dependencies beyond the C++ standard library, making it easy to integrate into any C++20 project. It is extensively tested with a vendored copy of Google Test and includes CI pipelines for multiple compilers and C++ standards (C++20 through C++26).