$ show zhaofengli/attic
Attic is a self-hostable Nix binary cache server written in Rust. It provides a multi-tenant, globally deduplicated cache for Nix store paths, supporting both local filesystem and S3-compatible storage backends. The system consists of a server binary (`atticd`) that implements the Nix binary cache HTTP API and a client CLI (`attic`) for authentication, pushing store paths, and managing caches. Authentication is stateless via signed JWTs (RS256/HS256) with fine-grained permissions per cache. The server uses content-defined chunking (FastCDC) to split NAR archives into variable-sized chunks, enabling deduplication at both the NAR and chunk levels. Compression (brotli, zstd, xz) is applied during upload with simultaneous hash computation for integrity verification. Garbage collection supports per-cache retention periods and cascading deletion from local caches to global chunk storage. The codebase is organized as a Cargo workspace with four crates: `attic` (core library with chunking, hashing, Nix store FFI), `server` (HTTP server, database, storage, garbage collection), `client` (CLI with push, watch-store, cache management), and `token` (JWT creation and verification). The architecture is modular, with clear separation between API handlers, database entities, storage backends, and authentication. While the project is still in early prototype stage with many TODOs and some fragile FFI bindings to Nix's C++ store library, it demonstrates solid engineering practices with reusable components like streaming hash readers, compression streams, and a custom serde serializer for Nix manifest format.
The LLM identified 4 files worth highlighting for reuse:
HashReader is an async read filter that computes a digest (e.g., SHA-256) while streaming data. It wraps any AsyncRead and yields the hash after reading completes. This is a clean, reusable pattern for any scenario requiring streaming data integrity verification, such as upload validation or content-addressable storage. It is self-contained and could be extracted into a separate crate with minimal changes.
CompressionStream wraps an async stream with a compressor (brotli, zstd, xz, or none) and simultaneously tracks both the compressed and uncompressed hashes and sizes. This enables the server to verify integrity of the original data after decompression without buffering the entire payload. The pattern is reusable for any application that needs to compress data on the fly while preserving original hash information, such as backup systems or content delivery networks.
PushPlan computes the closure of store paths to push, filters out paths already cached upstream or locally, and produces a plan for uploading missing paths. It handles batching, parallel uploads, and progress reporting. This component encapsulates a common pattern for incremental data synchronization with deduplication awareness. It could be adapted for other content-addressable storage systems that need to compute and upload only missing objects.
This module implements a custom serde serializer for the Nix manifest format, which is a flat key-value store with colon delimiters. It demonstrates non-obvious use of serde's Serializer trait to produce a non-standard text format. The implementation is self-contained and could be reused for any project that needs to serialize Rust structs into similar flat KV formats (e.g., environment files, simple configs).