$ show cloudflare/pingora
Pingora is a production-grade Rust framework for building fast, reliable, and programmable networked systems, developed and battle-tested at Cloudflare (handling 40M+ requests/second). It provides a modular architecture with crates for HTTP/1.1 and HTTP/2 proxying, TLS termination (OpenSSL, BoringSSL, rustls, s2n-tls), load balancing (weighted round-robin, consistent hashing), caching (LRU, TinyUFO), rate limiting, and observability (Prometheus, Sentry). The core abstraction is the `ProxyHttp` trait, which allows users to define request/response filters, upstream selection, error handling, and module integration via lifecycle hooks. The framework also includes a server lifecycle manager with graceful shutdown, daemonization, and file descriptor transfer for zero-downtime upgrades. The architecture is library-oriented: users compose services by implementing traits and configuring YAML-based server settings. Key entry points include `pingora::Server` for process management, `pingora_proxy::http_proxy_service` for creating proxy services, and `pingora_load_balancing::LoadBalancer` for background health checks and discovery. Data flows involve downstream HTTP sessions (H1/H2), upstream connections via pooled connectors, and optional caching layers with eviction policies. External dependencies include tokio, http, h2, lru, ahash, and various TLS libraries. While primarily an application framework, Pingora contains research-grade components like `tinyufo` (a lock-free cache algorithm with TinyLFU admission and S3-FIFO eviction) and `pingora-ketama` (nginx-compatible consistent hashing). These are packaged as reusable crates with extensive benchmarks and memory profiling. The codebase is well-structured with clean trait-based abstractions, but the large workspace (20+ crates) and multiple TLS backends introduce maintenance complexity. Overall, it is a high-quality, actively maintained framework suitable for building custom proxies, load balancers, and edge services.
The LLM identified 6 files worth highlighting for reuse:
Implements a concurrent, lock-free cache using TinyLFU admission and S3-FIFO eviction. Provides high hit ratios and throughput compared to LRU, Moka, and QuickCache. The algorithm is self-contained, with atomic operations and crossbeam queues, making it reusable as a standalone cache crate.
A sharded LRU cache with Power-of-Two-Choices (P2C) eviction for balanced shard usage. Supports variable-size keys, lock-free reads, and persistence via atomic temp-file writes. The design is generic and can be extracted for any concurrent caching need.
Implements a global cache lock manager that serializes cache fills for the same key, with timeout and fill publication. Readers can abandon wait if the writer publishes an unusable fill token, reducing wasted waiting. This pattern is reusable for any cache stampede protection scenario.
Provides a rich error type with cause chains, retry classification (upstream/downstream/internal), and context strings. Includes ergonomic traits like `Context`, `OrErr`, and `OkOrErr` for wrapping errors. This is a standalone, reusable error handling library.
Implements a Count-Min Sketch frequency estimator with atomic counters, aging via right-shift, and periodic window reset. Used for rate limiting and inflight tracking. The algorithm is self-contained and can be reused for any approximate counting problem.
A Rust port of nginx's consistent hash function (ketama) with memory-optimized V2. Includes backward-compatibility tests against real nginx output. The implementation is isolated and can be reused for distributed load balancing.