$ show misha-tgshv/shadowrocket-configuration-file
The repository is a data pipeline that generates Shadowrocket configuration files for selective traffic routing, particularly targeting users in Russia who need to bypass censorship while maintaining access to local services. It automates the collection, processing, and formatting of domain and IP lists from various sources into Shadowrocket-compatible rule sets and configuration files. The pipeline fetches data from diverse sources including GitHub repositories, websites (like the Central Bank of Russia), and manually curated lists. It then processes this data through a series of transformations: filtering, formatting, IDN conversion, section-based processing, and DNS resolution, to produce standardized output files that adhere to Shadowrocket's rule syntax. The processed data is used to generate configuration files that define which traffic should go through proxies and which should connect directly. Key reusable components within the pipeline include an IDN decoding script that properly handles internationalized domain names by detecting non-ASCII characters and converting to punycode while preserving original formatting, and a section-based list processor that applies different transformations to file segments based on markers. These components demonstrate non-obvious skill in handling specific data challenges and follow patterns that could be extracted for use in other data processing projects. The pipeline is executed on a daily schedule via GitHub Actions, ensuring the generated configuration files remain up-to-date with the latest domain and IP information from the source feeds. This automation allows users to benefit from continuously improved routing rules without manual intervention.
The LLM identified 2 files worth highlighting for reuse:
This script processes domain names from stdin, detects non-ASCII characters indicating potential internationalized domain names, converts them to punycode format while preserving the original in a comment, and outputs results. It handles URL decoding, splits domains by dots, processes each part, and includes error handling. It is self-contained, uses only standard Python libraries, and solves a recurring IDN handling problem that could be extracted into any Python-based data processing project with minimal modification. It demonstrates the pattern of robust internationalized domain name handling including proper detection of non-ASCII characters, accurate punycode conversion, and preservation of original formatting for readability.
This awk script processes lists with section markers, applying different transformations to entries before and after a specified section marker ('# youtube'). It prints comment lines as-is, skips empty lines, and for regular lines applies 'DOMAIN-KEYWORD,' prefix before the marker and 'DOMAIN-SUFFIX,' prefix after the marker. It also generates a standardized header with metadata including timestamp and source information. It is self-contained awk code that solves the general problem of processing sectioned files with section-specific transformations. It could be adapted to other sectioned files by changing the section marker and transformation rules, making it a versatile tool for configuration file and rule list processing. It demonstrates the pattern of section-based data processing where different transformation rules apply to different sections of a file, using awk's ability to maintain state across lines and apply conditional logic based on section boundaries.