$ show ermaozi/get_subscribe
This repository is a proxy subscription aggregation and distribution system. It periodically collects free proxy subscription links (Clash YAML and V2Ray URI formats) from multiple public sources, merges them, classifies nodes by TCP connectivity (reachable/unreachable/untested), and serves the resulting files via a Cloudflare Worker. The system is driven by a GitHub Actions workflow that runs every 12 hours, executing Python scripts to fetch, process, and commit updated subscriptions. The architecture is a data pipeline application with a serverless serving layer. Key entry points include main.py (orchestrating collection and merging), check_nodes.py (TCP connectivity classification), and worker/sub.mjs (Cloudflare Worker for serving with caching and ETag). External dependencies include requests, PyYAML, and the Cloudflare Workers runtime. The repository also includes a weekly commit history cleanup workflow to keep repository size small. The project is actively maintained with frequent updates and tests, though it relies on unstable external proxy sources.
The LLM identified 4 files worth highlighting for reuse:
Contains `endpoint()` which extracts TCP host/port from various proxy formats (vmess, ss, ssr, vless, trojan, http/socks) and `tcp_status()` which performs TCP connect with timeout, skips private addresses, and returns reachable/unreachable/untested. This module is self-contained and can be extracted for any project needing to test proxy endpoint reachability.
Provides `_build_session()` (requests.Session with retry adapter and optional proxy), `_detect_kind()` (heuristic detection of Clash YAML vs V2Ray URI format), `_classify_subscriptions()` (downloads candidate URLs and identifies valid subscriptions), and `_merge_clash()` (merges multiple Clash configs with deduplication by identity hash, name collision resolution, and group renaming). These functions are reusable for any subscription aggregation pipeline that needs to merge proxy configurations.
Implements a Cloudflare Worker that syncs subscription files from GitHub to R2 bucket (`refreshAll`) and serves them with ETag, cache headers, and custom headers. The pattern of fetching from GitHub raw, validating, storing in R2, and serving with conditional requests is reusable for any project needing a serverless file distribution layer.
Provides unit tests for the subscription collection and merging logic, including format detection, deduplication, and classification. The test structure is well-organized and can serve as a template for testing similar subscription aggregation pipelines.