$ account nlfiedler
This is a pure Rust library implementing the FastCDC (Fast Content-Defined Chunking) algorithm, providing three independent implementations: ronomon (legacy, 31-bit), v2016 (original paper), and v2020 (optimized with 'rolling two bytes each time'). The v2020 module is the recommended entry point, offering in-memory chunking via `FastCDC` (Iterator), synchronous streaming via `StreamCDC`, and async streaming via `AsyncStreamCDC` (with tokio/futures feature flags). The core algorithm uses Gear hash tables derived from MD5 digests, normalized chunking with configurable levels, and an unrolled inner loop that processes six bytes per iteration for performance. The crate is deterministic: identical inputs always produce identical cut points and hashes, enforced by hardcoded fixture tests. It is designed for deduplication systems, backup software, and any application needing content-defined chunk boundaries. The codebase is well-structured, thoroughly tested, and includes a criterion benchmark suite with deterministic data generation for reproducible performance comparisons.