$ show jkwakman/Open-Cookie-Database
The Open Cookie Database is a community-curated reference dataset cataloging HTTP cookies. The single source of truth is open-cookie-database.csv, which records each cookie's platform, category (Functional, Personalization, Analytics, Marketing, Security), cookie name/pattern, domain, description, retention period, data controller, GDPR/privacy policy URL, and a wildcard-match flag. The dataset is maintained through manual PR-based contributions following a documented research workflow, and it is consumed by downstream privacy tooling (notably the EU EDPB Website Auditing Tool, via a dedicated EDPB JSON export). A static HTML viewer renders the CSV client-side using PapaParse and DataTables for browsing and export. The data lineage is simple and fully automated: on every push touching the CSV, GitHub Actions runs validate.py (UUID format/uniqueness, category whitelist, duplicate detection), then regenerates two derived artifacts - open-cookie-database.json (records grouped by platform) via convert_csv_to_json.py, and open-cookie-database-edpb.json via convert_edpb.py, which maps the flat CSV onto the EDPB tool's nested knowledge-base JSON schema. The pipeline is deterministic, idempotent, and commits the regenerated JSON back to the repo when changed. There is no application runtime, API, or library; the three Python scripts and CI workflows are infrastructure that exists solely to keep the dataset valid and in sync across formats. Craftsmanship is functional but modest. The scripts show reasonable engineering discipline: CLI parameterization, graceful handling of missing/empty files, GitHub Actions error annotations, and non-zero exit codes on failure. The validation script encodes real data-quality rules. However, there are clear signs of aging: the HTML viewer references the retired StackPath Bootstrap CDN and contains a malformed protocol-relative script URL ('//http://...'); the main validation workflow still pins Node 16 (EOL) and uses checkout@v2; convert_edpb.py uses the deprecated datetime.utcnow(); and pandas is installed unpinned. These issues do not break the core CSV-to-JSON pipeline, but they make the supporting surface fragile. The repository is a data asset first and code second - the value lies in the curated cookie catalog and the governance around it, not in the software. It is a solid, genuinely useful open-data project with competent but unremarkable data engineering. An analyst can clone it and reproduce all derived JSON artifacts with just Python and pandas, since the dataset is bundled and the transforms are simple, explicit, and parameterized.
The LLM identified 3 files worth highlighting for reuse:
A self-contained CSV schema validator enforcing three common data-quality rules: IDs must be valid unique UUIDs, categorical columns must match an allowed-value whitelist, and natural keys must not be duplicated. It emits GitHub Actions-compatible error/warning annotations and prints the offending rows for triage. The pattern - validate primary key integrity, constrain categorical values, and warn on near-duplicates - transfers directly to any CSV-published dataset that needs structural guarantees before it is released to consumers.
A parameterized CSV-to-grouped-JSON converter that flattens row records into a nested structure keyed by a grouping column (here, Platform). It is robust to missing input files, empty CSVs, and absent required columns, returning distinct error paths and GitHub Actions annotations for each failure mode. The column-mapping dictionary is the only project-specific layer, so the script can be adapted to any flat tabular dataset needing a grouped/nested JSON export with minimal modification. Demonstrates a clean, idempotent ETL helper pattern backed by CI triggers.
Shows how to adapt a neutral canonical CSV into a vendor-specific, nested JSON schema - in this case the EU EDPB Website Auditing Tool's knowledge-base format. Features include mapping flat columns to a nested 'knowledges' array, normalizing category strings to lowercase, coercing the wildcard flag to an integer with a safe fallback, and stamping consistent ISO-8601 timestamps on every entry. This is a reusable adapter pattern for any project that must serve one dataset to multiple downstream consumers with incompatible schemas. Note: it uses deprecated datetime.utcnow(), which should be replaced with datetime.now(timezone.utc) for Python 3.12+.