$ show Cysharp/MemoryPack
MemoryPack is a high-performance binary serialization library for .NET (C#) that achieves extreme speed through 'zero encoding'—direct memory copies for blittable types rather than varint/tag-based encodings. It is primarily a library, distributed as multiple NuGet packages, including a Roslyn source generator that compiles serialization code for types marked with [MemoryPackable]. The core library (MemoryPack.Core) provides writer/reader ref structs, a customizable formatter architecture, and support for compression (Brotli), streaming (PipeWriter/PipeReader), version-tolerant and circular-reference serialization, and polymorphism via union types. It integrates with ASP.NET Core via input/output formatters, generates TypeScript/JavaScript serializers for web clients, and ships a Unity package with shims for common Unity types. The codebase is extensively benchmarked and tested, focusing on allocation-free, high-throughput serialization. While not a data research project, it may be used in data pipelines requiring fast, compact binary data exchange.
The LLM identified 4 files worth highlighting for reuse:
Implements a compression layer over Brotli that exposes IBufferWriter<byte> for efficient serialization. Can be extracted as a general-purpose pattern for wrapping any System.IO.Compression stream as a buffer writer/receiver, enabling zero-copy compression during serde. Demonstrates non-obvious skill: bridging Stream and IBufferWriter with proper disposal and flush semantics.
Counterpart to BrotliCompressor, wrapping BrotliStream decompression over ReadOnlySequence<byte>. Provides the decompression side of the same buffer-oriented pattern, useful for integrating compressed binary formats into any pipeline that consumes spans.
A pooled, reusable IBufferWriter<byte> implementation that chains linked arrays to grow without internal resizing. Reduces allocation pressure for large or unpredictable serialization payloads. The design pattern—returning a pooled resource, resetting state, and using linked segments—is applicable to any high-throughput buffering scenario.
Integrates MemoryPack deserialization into ASP.NET Core MVC's input pipeline. Demonstrates a clean, minimal implementation of IInputFormatter that reads binary payloads from the request body. The pattern is reusable for any custom binary format in ASP.NET Core, showing correct media-type negotiation and stream management.