Protocol Buffers is a language-neutral, platform-neutral, extensible mechanism for serializing structured data. It provides a protocol compiler (protoc) that translates .proto source files into language-specific code, along with runtime libraries for multiple languages (C++, Java, Python, etc.) to serialize and deserialize structured data. The architecture is modular, separating the compiler, language runtimes, and code generators, with a focus on high-performance serialization and cross-language compatibility. The system is used in production for data storage, RPC, and inter-process communication across a wide range of applications. Key entry points include the protoc compiler and the language-specific runtime libraries. Data flows involve defining schemas in .proto files, generating code, and then using the generated classes to convert between in-memory objects and binary or text (JSON) representations. The project relies on external dependencies such as Abseil and googletest for its C++ implementation, while maintaining its own lightweight upb implementation for certain language bindings. While the repository is primarily a software development kit (SDK) for data serialization, it also includes extensive testing, benchmarking, and conformance suites to ensure correctness across language implementations. This dual focus on being a usable library and maintaining strict correctness places it firmly in the 'application' category, with no significant data research components.
gRPC is a high-performance, open-source universal RPC framework that enables client and server applications to communicate transparently and simplifies the building of connected systems. It provides language bindings for C++, Python, Ruby, PHP, Objective-C, and C#, uses protocol buffers for interface definition and serialization, and supports HTTP/2 transport with advanced features like authentication, load balancing, tracing, and health checking. The architecture consists of a core C++ implementation handling transport and channel management, language-specific wrappers, and a plugin/extension system for customizing behavior.
Tachyon is a high-performance, cross-language IPC library that provides a lock-free, single-producer single-consumer (SPSC) ring buffer over POSIX shared memory. It achieves sub-100 nanosecond round-trip times by using zero-copy payloads, futex-based blocking, and a handshake protocol that transfers anonymous memory file descriptors via Unix domain sockets. The library exposes a C API and ships idiomatic bindings for Python, Rust, Go, Java, Kotlin, C#, and Node.js/TypeScript, all sharing the same core C++ implementation. It supports both unidirectional publish-subscribe and bidirectional RPC patterns, with batch operations, NUMA binding, and configurable polling modes. The codebase is production-grade: it includes extensive fuzz testing, architecture decision records, seccomp BPF profiles, a monitoring CLI tool, and a mature release pipeline publishing to all major package registries. The design is notable for its minimal syscall footprint on the hot path (only futex), careful cache-line alignment, and a no-serialization contract that lets consumers read payloads directly from shared memory.
This repository defines the Transit data interchange format (v0.8), a cross-language serialization specification for conveying values between applications. It does not contain any implementation code for reading or writing Transit; instead, it provides the authoritative specification (README.md), a comprehensive suite of example data files in multiple representations (EDN, JSON, verbose JSON, MessagePack), and a Clojure-based testing infrastructure to verify that third-party Transit implementations correctly encode/decode according to the spec. The test harness in `src/transit/verify.clj` orchestrates roundtrip tests by invoking external implementation processes (via `bin/verify`), using both hand-crafted corner cases and generative random data to ensure correctness. The repo acts as the central reference for all Transit implementations across languages including Clojure, Java, JavaScript, Python, Ruby, and others.