$ show FiloSottile/sunlight
Sunlight is a production-grade, static Certificate Transparency (CT) log implementation written in Go, sponsored by Let's Encrypt. It implements the C2SP Static CT API and provides both a write-path log server (cmd/sunlight) and a read-path static file server (cmd/skylight). The system accepts certificate submissions via the RFC 6962 add-chain/pre-chain endpoints, sequences them into a Merkle tree using the tlog library, and publishes tiles (hash, data, names) along with signed checkpoints. It also supports the tlog-witness and tlog-mirror protocols for transparency and cross-checking. The architecture is modular: the core log engine lives in internal/ctlog with pluggable backends for storage (local filesystem, S3, in-memory) and distributed locking (SQLite, DynamoDB, ETag-based S3). The sequencer uses a pool of pending entries with low-priority eviction, batched sequencing, and staged uploads via tarballs for crash recovery. Deduplication is handled via a SQLite cache with 256-bit hashes. The system includes extensive observability with Prometheus metrics, structured logging, and debug endpoints for heavy hitters and SSL key logging. Key entry points include the main server binaries (cmd/sunlight, cmd/skylight), a key generation tool (cmd/sunlight-keygen), a partial tile garbage collector (cmd/partial-aftersun), a cache recomputation tool (cmd/recompute-cache), and a names tile generator (cmd/names-prism). The client library (client.go) provides a high-level API for fetching and verifying log entries and inclusion proofs. External dependencies include certificate-transparency-go, torchwood, and golang.org/x/mod/sumdb/tlog. The codebase demonstrates high craftsmanship with several reusable internal packages: durable crash-safe file writes, immutable file flag setting, HyperLogLog cardinality estimation, Space Saving heavy hitters, and a custom XAES-256-GCM implementation. Testing is extensive, including in-memory fake backends, synctest for time-dependent rate limits, and deterministic test data generation.
The LLM identified 5 files worth highlighting for reuse:
Implements WriteFile and MkdirAll with crash-safe semantics: writes to a temporary file, fsyncs, renames to target, then fsyncs the parent directory. This ensures atomicity and durability even in the event of a system crash. The pattern is self-contained and can be extracted into any Go project that needs reliable file writes, especially for databases or log storage.
Provides a HyperLogLog sketch for distinct count estimation, including a sliding-window variant for time-decayed cardinality. The implementation is clean, well-tested, and uses minimal dependencies. It can be reused in any Go project requiring approximate distinct counting, such as analytics pipelines or network monitoring.
Implements the Space Saving algorithm for finding frequent items in a data stream with bounded memory. The implementation is thread-safe and provides a simple API for tracking top-k items. This is a classic streaming algorithm that can be reused in any Go application needing heavy hitter detection, such as traffic analysis or log monitoring.
Implements XAES-256-GCM, an AES-256-GCM variant with an extended 192-bit nonce using a derived sub-key approach. The implementation is self-contained, well-documented, and includes test vectors. It demonstrates non-obvious cryptographic engineering and can be reused in any Go project needing a nonce-misuse-resistant AEAD scheme.
Bridges the RFC 6962 Signed Tree Head (STH) format with the tlog note-signed checkpoint format. It provides functions to convert between the two, inject signers/verifiers, and handle dual signatures (ECDSA + ML-DSA). This is a critical component for interoperability between legacy CT logs and the static CT API. The pattern of adapting signature formats is reusable in any transparency system.