$ show EpicWink/proxpi
proxpi is a caching reverse-proxy server for PyPI and any PEP 503-compatible package index. It fronts one or more upstream indices, serves the Python simple repository API to pip clients in both HTML (PEP 691 v1.0) and JSON (PEP 691 v1.1) forms, and performs correct HTTP content negotiation including Accept, Accept-Encoding, and Vary headers. Index responses (project list and per-project file lists) are cached in memory with per-index TTLs; package files are cached on disk with a configurable size cap and least-frequently-used eviction. When a requested file is still downloading in the background, the server emits a 302 redirect to the upstream URL rather than blocking the client. The system is a single-process Starlette ASGI application (proxpi.server:app) designed for multi-threaded serving under Gunicorn/Hypercorn, configured entirely through environment variables (index URLs, TTLs, cache size/dir, connect/read/download timeouts, SSL verification toggle, log level). Its main target is reducing pip install latency and upstream load in CI pipelines. The implementation is a compact three-layer design: server.py handles routing, templating, and compression; _cache.py holds the caching engine (a File abstraction with FileFromHTML/FileFromJSON parsers for the dual-format simple repository API, an _IndexCache with per-package locks and a fast-path that fetches project pages without refreshing the full index, and a _FileCache with background downloads, atomic os.replace writes, and LFU eviction); _server_utils.py provides RFC-correct content-negotiation helpers built on the stdlib email header parser. The code is thread-safe by design (per-key locks, exception-storing worker threads) and memory-efficient, streaming HTML index parsing through lxml.iterparse via a custom file-like response adapter. The test suite is notably strong: parametrized integration tests against mock HTML/JSON indices covering yanked files, core-metadata hashes, gpg-sig attributes, content negotiation edge cases, cache invalidation, and 404 paths, plus end-to-end tests that run real pip downloads against live PyPI. CI runs the matrix across Python 3.8-3.14, lints with black/isort, and publishes multi-arch Docker images with fully hash-pinned dependencies. One caveat: the README is stale, documenting the pre-1.3 Flask implementation, while the code has already migrated to Starlette (listed in CHANGELOG as unreleased), and the Docker image now runs Gunicorn with an ASGI worker.
The LLM identified 2 files worth highlighting for reuse:
Correct RFC 9110 HTTP content negotiation utilities. parse_accept_header and parse_accept_encoding_header parse their headers using the stdlib email.headerregistry media-type parser, preserving parameters and q-values, then return memoized (lru_cache) closures that score a candidate media type against wildcard rules (type/*, */*, subtype matching, parameter matching). This is non-obvious because naive comma-splitting Accept parsers mishandle q-values, media-type parameters, and wildcard precedence. add_vary appends Vary header names without duplication. The module has zero runtime imports beyond stdlib (Starlette types appear only under TYPE_CHECKING), so it can be dropped into any Python web framework or HTTP client library unmodified.
A complete, self-contained package-index caching engine with no internal-module dependencies (only requests, lxml, and stdlib). The File abstract base class with FileFromHTML/FileFromJSON implementations cleanly abstracts the PEP 691/700/714 dual-format simple repository API, normalizing yanked status, core-metadata hashes, gpg-sig markers, and PEP 714 key migration. _IndexCache implements TTL caching with a per-key lock registry (_Locks) and a clever fast-path: when the index TTL is expired it requests the project page directly, only falling back to listing all projects if that fails. _FileCache provides a disk cache with least-frequently-used eviction, background-thread downloads deduplicated via a thread-in-dict pattern, atomic file publication via os.replace of a .proxpi-partial suffix, and a graceful timeout that redirects clients to the source URL while a download is in flight. The _ResponseReader adapter wraps requests.Response.iter_content into a file-like .read(n) interface so lxml.etree.iterparse can stream-parse large index HTML without loading the full body. The whole module could be extracted as-is to build a caching proxy for any HTTP-served index API.