$ show ijl/orjson
orjson is a high-performance JSON serialization and deserialization library for CPython, implemented as a Rust extension using PyO3 with direct Python C-API FFI. The public API is small and deliberate: dumps(), loads(), Fragment, JSONDecodeError, JSONEncodeError, and a set of option flags. Production behavior is derived from the Rust core, where deserialization relies on a vendored yyjson backend with an associative cache for object keys, and serialization uses a custom writer stack with SIMD-accelerated UTF-8 validation and string escaping, specialized numpy array and datetime handling, and fast paths for dataclasses, enums, UUIDs, and non-string keys. Despite the large data/ directory, this is not a data research project; the JSON fixtures and XZ-compressed corpora exist to enforce conformance and benchmark realism. The architecture is a classic C-extension pattern: src/lib.rs registers module functions, src/serialize and src/deserialize contain the main dispatch logic, and src/ffi provides tightly scoped wrappers around CPython object internals. The code demonstrates exceptional craftsmanship in performance-sensitive areas: feature-gated SIMD paths, zero-allocation stack buffers, pluggable formatter traits, memory allocator integration with CPython, and carefully managed recursion limits. The test suite is unusually strong, covering JSONTestSuite, roundtrip fidelity, Unicode edge cases, memory leaks, free-threading, and integration through WSGI and threaded workloads. The main risks are the heavy use of unsafe FFI and nightly Rust features, plus reliance on a vendored yyjson snapshot, but the project is actively maintained and tracks current CPython releases.
The LLM identified 5 files worth highlighting for reuse:
A self-contained, compile-time configurable JSON reader/writer in C with immutable and mutable document models, incremental streaming parsing, JSON Pointer/Patch/Merge Patch, and pluggable allocators. Although vendored, its public API is standalone and can be lifted directly into any C/C++ project. The implementation demonstrates advanced parser engineering: FSM-based root parsing, correctly rounded Schubfach float conversion, careful endian/unaligned memory handling, and macro-driven loop unrolling.
Feature-gated JSON string escaping and UTF-8 validation with AVX-512, SSE2, generic-SIMD, and scalar implementations behind a common interface. This is a reusable performance pattern for any high-throughput text serializer: the selector picks the best available vectorized path, and each backend handles escaping, surrogate pairs, and invalid-byte rejection without allocating. It requires Rust intrinsics expertise to extract, but the dispatch structure is clean and instructive.
An associative key cache for JSON object keys during deserialization, using Capacity2048, HashDirectMapped, and RoundRobinReplacement. It avoids repeated key string allocation and hashing in hot parse loops. The cache is isolated and demonstrates a concrete, reusable technique for any key-value parser that wants to amortize key lookup costs across repeated documents.
A fixed-capacity stack buffer designed for serializing fixed-size values such as UUIDs and datetimes without heap allocation. It is a small, self-contained low-latency write path that can be extracted into any serializer or formatter needing temporary fixed-length output. The pattern is especially valuable for avoiding allocator pressure in tight loops.
Hundreds of JSON conformance fixtures covering the JSONTestSuite and nativejson-benchmark edge cases: invalid UTF-8, surrogate handling, numeric formats, deep nesting, BOMs, and malformed structures. This corpus is a reusable asset for validating any JSON parser implementation, and the repository's script/pycorrectness demonstrates how to classify fixtures into must-pass and must-fail expectations.