$ show everywall/ladder
Ladder is a self-hosted HTTP web proxy written in Go, designed for testing and analyzing paywall implementations, content delivery, and access control behavior on modern websites. It intercepts client requests, applies domain-specific YAML rulesets to modify request headers, URLs, and response content (e.g., removing CORS/CSP headers, injecting scripts, rewriting relative URLs), then fetches the target site and returns the modified response. The proxy exposes a web form, a JSON API, a raw HTML endpoint, and a ruleset endpoint, and supports optional FlareSolverr integration for bypassing Cloudflare challenges. It is deployable as a binary, Docker container, or Helm chart, and is configured via environment variables and CLI flags. The architecture is a single Go binary using the Fiber web framework. The main entry point (cmd/main.go) sets up routing, basic auth, and middleware. Core logic lives in handlers/proxy.go, which handles URL extraction, ruleset application, HTTP fetching, HTML rewriting, and FlareSolverr integration. The pkg/ruleset package provides a clean, reusable ruleset loader supporting local files, directories, remote URLs, and gzip-compressed YAML. The codebase is generally well-structured with some global state and a few misnamed test files (e.g., proxy.test.go instead of proxy_test.go) that are not executed by Go's test runner. Despite these minor issues, the project is actively maintained, uses current dependencies, and demonstrates solid engineering practices for a focused tool.
The LLM identified 2 files worth highlighting for reuse:
A self-contained Go package for loading, merging, and serializing YAML-based rulesets from local files, directories, remote URLs, and gzip-compressed sources. It provides a clean API (NewRuleset, NewRulesetFromEnv, Yaml, GzipYaml, Domains) and robust error handling with clear warnings and panics for missing local files. This package is reusable in any project needing a flexible, multi-source configuration loader and demonstrates non-obvious skill in handling remote and compressed config sources.
The extractUrl function resolves relative URLs in a proxy context by using the Referer header to reconstruct the full URL, handling base path prefixes and rejecting unresolvable paths with descriptive errors. This pattern is essential for any reverse proxy that must handle relative asset requests, and the logic is isolated enough to be extracted into a utility function for reuse in similar proxy implementations.