$ account xroche
HTTrack is a mature, production-grade website mirroring/offline browsing tool written in C. It recursively downloads websites to a local directory, rewriting links to enable offline browsing. The system includes a command-line interface (httrack), a web-based GUI (htsserver), a caching proxy (proxytrack), and a shared library (libhttrack). The core engine manages multi-threaded HTTP/HTTPS/FTP downloads, incremental updates via a zip-based cache, WARC/WACZ archival output, and robust link filtering. The architecture is monolithic but modularized into loosely coupled subsystems (caching, URL naming, HTML parsing, content decoding, robots.txt handling, etc.). Key entry points are in src/httrack.c, src/htsserver.c, and src/proxy/main.c. Data flows: user-provided URL seeds → DNS resolution → HTTP fetch → HTML parsing → link extraction → deduplication → resource download → local mirror tree or WARC output. External dependencies include zlib (required), OpenSSL (optional for WARC digests), and iconv (optional for charset conversion). The codebase is extensively tested with integration tests, fuzz harnesses, and self-tests; it also includes a standalone reusable test HTTP server and a WARC validator.