$ show datasets/country-codes
This repository is a data engineering pipeline that produces a comprehensive, unified country codes dataset (CSV/JSON) by merging information from multiple authoritative sources: ISO 3166, ISO 4217, UN Statistics, UN Protocol (UNTERM), CLDR, statoids, Geonames, EDGAR, FIFA, and Wikidata. The pipeline is orchestrated via a Makefile that sequentially runs Python scripts to fetch, parse, clean, and join data. The final output contains dozens of columns covering country names in multiple languages, ISO codes, currency codes, dialing codes, and more. The repository is primarily a data transformation project rather than a software application, though it includes sophisticated web scraping and fuzzy matching. The code is modular and well-structured, but reproducibility is hampered by a manual step requiring users to manually download and rename UNSD CSV files. The included GitHub Actions workflow automates monthly updates. Overall, the project is a solid reference data curation effort with moderate complexity.
The LLM identified 2 files worth highlighting for reuse:
Implements a curated list of field corrections that survive pipeline reruns. Each correction records the match column, value, target field, correct value, reason, and source. This pattern solves a common data engineering problem: upstream errors that are faithfully reproduced on every pipeline run. The component is self-contained, uses only csv and no internal imports, and can be extracted into any data pipeline that needs to apply persistent field-level overrides after automated processing.
Fetches the authoritative list of FIFA member codes from inside.fifa.com via a single HTTP request, extracting codes from anchor hrefs. It then compares the existing FIFA column in the CSV with the valid set, and falls back to the ISO 3166-1 alpha-3 code if the current value is not valid. This pattern is reusable for any project that needs to validate and correct a code column against an authoritative source, with a cross-reference fallback strategy. The script is self-contained with no internal imports.