$ show ahamez/protox
Protox is an Elixir library for Google Protocol Buffers (protobuf) that provides encoding/decoding to/from binary wire format, code generation from .proto schemas, and compile-time schema compilation. It supports both proto2 and proto3 syntaxes. The library is used by defining a module with `use Protox` and providing either an inline schema string or a list of .proto files; this macro triggers protoc to parse the definitions and generates Elixir structs with encode/decode functions, unknown field handling, and schema introspection. The generated code is self-contained and only requires the protox runtime for encoding/decoding primitives. The architecture is a mix of compile-time code generation (via Elixir macros) and runtime encoding/decoding primitives. Key entry points include the `Protox` macro, `Protox.encode/1`, `Protox.decode/2`, and the `mix protox.generate` task for generating standalone source files. The library relies on `protoc` (the protobuf compiler) only at compile time to produce a binary descriptor; all runtime operations are pure Elixir. External dependencies are minimal: protoc must be in PATH during development, and the runtime has no non-OTP dependencies. Protox emphasizes correctness and performance: it passes Google's official conformance test suite, uses property-based testing with StreamData, mutation testing with Muex, and maintains near-100% test coverage. The encoding/decoding paths are heavily optimized with unrolled varint loops, inline binary pattern matching, and careful accumulator management to minimize allocations. The library also supports unknown fields, message merging, proto2 required fields, nested extensions, and proto3 optional fields.
The LLM identified 5 files worth highlighting for reuse:
Provides LEB128 variable-length integer encoding and decoding with unrolled clauses for each encoded length (1-10 bytes). The encode/1 function uses pattern matching on value ranges to produce exact-sized binaries without intermediate allocations. The decode/1 function uses compile-time generated clauses from DecodeClauses to avoid tuple construction per element. The append/2 function allows building packed binaries without per-element intermediate binaries. This module is self-contained (only depends on Bitwise and its own DecodeClauses) and could be extracted for any binary protocol using LEB128.
A compile-time helper that generates unrolled pattern-matching clauses for LEB128 decoding. It takes a range of encoded lengths and a rest variable, and produces {pattern, value} pairs that match varint bytes directly in binary patterns. This eliminates the need for a generic loop and tuple construction per decoded element. The pattern is reusable for any binary protocol that uses LEB128 or similar variable-length encoding, and demonstrates advanced Elixir metaprogramming for performance optimization.
Validates UTF-8 strings according to protobuf specifications, including a configurable maximum size limit (default 2^32 bytes, reduced to 2^20 in test mode for faster property tests). It uses `String.valid?/1` with `:fast_ascii` optimization for ASCII-heavy strings, and checks size before content to avoid walking oversized binaries. The module is self-contained and could be reused in any Elixir project needing protobuf-compliant string validation.
Defines guard macros for classifying protobuf scalar types into varint, fixed32, fixed64, and delimited categories. These guards are used throughout the encoding and decoding code to dispatch on wire types at compile time. The module is self-contained and could be extracted as a reusable type classification utility for any protobuf implementation in Elixir.
Generates random protobuf messages for property-based testing. It walks the schema recursively, using a pluggable `Values` behaviour to control value distributions (e.g., edge-case-heavy for tests, production-like for benchmarks). Handles oneofs, maps, repeated fields, nested messages, and well-known types. This module demonstrates a clean separation of schema traversal from value generation and could be extracted as a standalone test helper for any protobuf library in Elixir.