$ show psf/cachecontrol
CacheControl is a Python library that adds HTTP caching to the popular `requests` library. It ports the caching algorithms from httplib2, providing a transparent caching layer for HTTP sessions. The library intercepts HTTP requests and responses, caching responses based on standard HTTP caching headers (Cache-Control, Expires, ETag) and optional heuristic rules. It supports multiple cache backends including in-memory dictionary, file-based cache (with separate body storage for memory efficiency), and Redis. The architecture is modular: a `CacheController` implements the core caching logic (freshness calculation, conditional requests, cache invalidation), while `CacheControlAdapter` integrates with requests' transport adapter system. The library also provides a CLI tool (`doesitcache`) for testing caching behavior. It is well-tested, type-annotated, and follows modern Python packaging practices.
The LLM identified 4 files worth highlighting for reuse:
Implements the core HTTP caching logic: URL normalization, cache-control header parsing, freshness lifetime calculation (max-age, Expires, heuristic), conditional request headers (If-None-Match, If-Modified-Since), and cache response storage/update. This is the central algorithm that can be reused in any HTTP caching system, not just requests.
Provides a CallbackFileWrapper that wraps a file-like object, teeing all read data into a temporary file. When the underlying file is closed, it invokes a callback with the buffered data (as a memoryview via mmap). This pattern is useful for caching streaming responses without holding all data in memory, and can be extracted for any scenario requiring deferred processing of streamed data.
Handles serialization and deserialization of HTTP responses using msgpack with a versioned format (cc=4). It separates metadata from body (supporting separate body storage), handles Vary header matching, and reconstructs urllib3 HTTPResponse objects. The versioned approach allows backward compatibility and can be adapted for other serialization needs.
Defines a base class for caching heuristics that can modify response headers before caching. Includes built-in implementations: OneDayCache (adds 1-day Expires), ExpiresAfter (configurable delta), and LastModified (RFC 7234 Section 4.2.2 heuristic, 10% of age difference capped at 24h). The pattern allows users to inject custom caching strategies for servers that don't provide explicit cache headers.