$ show Guang000/Awesome-Dataset-Distillation
This repository is a comprehensive, curated bibliography for the field of dataset distillation, maintained by three researchers. It aggregates over 400 papers organized by methodology (gradient matching, distribution matching, generative approaches, etc.) and application domain (continual learning, privacy, medical, federated, graph, NLP, video, etc.). The project provides a GitHub Pages website (index.html) that renders the paper list with links to code, project pages, and BibTeX citations, plus a sidebar for navigation. Supporting Python scripts (transform_record.py, validate_articles.py, catch_latest.py) manage the data pipeline: parsing a custom update format, validating article records, and merging latest additions. The repository is a hybrid of a data curation project (structured paper database) and a static web application (client-side rendering). While the code is simple and functional, there are data quality issues such as trailing whitespace in URLs, mismatched citation keys, and future-dated entries, indicating manual update processes that are not fully automated. The project is actively maintained with frequent additions, but the lack of pinned dependencies and automated CI validation limits reproducibility.
The LLM identified 5 files worth highlighting for reuse:
Parses a custom delimiter-separated update file (data/update.txt) into structured JSON article records. Uses only standard library modules (json, os, re). The pattern of reading a flat file with a known delimiter and transforming it into a structured format is reusable for any similar manual curation workflow. The script is self-contained and could be adapted to other delimiter-based data ingestion tasks.
Validates article records from a JSON file against required fields (title, author, url, cite) and checks URL formats, cite-name regex, and existence of corresponding BibTeX files. Uses only standard library modules. This pattern of schema validation with exit-code semantics is reusable for any structured data curation project, especially when ensuring data integrity before publication.
Merges a list of latest paper names (data/latest_names.jsonl) with the main article database (data/articles.json) to produce a latest.json file grouped by date. Uses only standard library modules. The pattern of incremental updates by merging a small changelog with a master dataset is reusable for any curated list that needs to track recent additions without full rebuilds.
A self-contained JavaScript file that fetches JSON data files (articles.json, navi_abbr.json, latest.json, acknowledgements.json) and renders the entire paper list, navigation sidebar, latest updates section, and acknowledgements on the GitHub Pages website. It also fetches the GitHub star count via API. The pattern of a data-driven, client-rendered static site is reusable for any awesome-list or curated directory that wants a dynamic frontend without a backend.
The central structured database of papers, organized hierarchically by methodology and application. Each entry contains fields: title, author, url, github, cite, website. This schema is a reusable template for any curated paper list or bibliography. The hierarchical categorization (main vs applications, sub-categories) provides a proven organizational pattern that can be adapted to other research fields.