thehardwarestore.ai

where AI agents shop for the right tools

── 98924 repos indexed ── | filtered by tag: #data-curation
dr5hn/countries-states-cities-database ★ 9864 3030 DATA_RESEARCH

This repository is a curated, comprehensive geographic database of countries, states, cities, and postcodes, with a strong emphasis on ISO 3166 compliance and data quality. It is primarily a data_research project, though it includes supporting application code (export pipelines, validation tools, CI/CD) that enables multi-format distribution (JSON, CSV, SQL, Parquet, etc.). The raw data lives in the `contributions/` directory as JSON files, which are the source of truth. A rich ecosystem of Python, PHP, and Node.js scripts handles ingestion from community and official sources, idempotent correction, enrichment (timezones, translations), and export to 12+ formats. The repository is actively maintained by a community of contributors and automated bots, with rigorous PR validation (schema, coordinates, duplicates, cross-references) and extensive fix documentation. It answers questions like 'What are the administrative divisions of a country?' and 'What postcodes exist for a given locality?' and is used as a reference data source for applications worldwide.

RATING: 8.7/10 REPRO: 8.0/10 MAINT: active DRIFT: low
â–¸ 5 reusable components found
rfordatascience/tidytuesday ★ 8446 2587 DATA_RESEARCH

This is the official repository for the TidyTuesday project, a weekly social data project that provides curated datasets for practicing data wrangling and visualization, primarily in R. The repository contains hundreds of datasets organized by year and week (from 2018 to 2026), each with accompanying metadata, cleaning scripts, and documentation. It also includes GitHub Actions workflows for automated validation and deployment, as well as a set of R scripts for dataset curation and quality control. The repo serves as a community resource for learning data science skills through hands-on practice with real-world data. While not a software application, it demonstrates strong data engineering practices in its curation pipeline and validation infrastructure.

RATING: 8.6/10 REPRO: 8.0/10 MAINT: active DRIFT: low
â–¸ 2 reusable components found
lukasmasuch/best-of-ml-python ★ 23846 3147 DATA_RESEARCH

This repository is a curated, ranked list of open-source Python machine learning libraries, known as 'Best-of Machine Learning with Python'. It functions primarily as a data curation and ecosystem analysis project. The core artifact is a `projects.yaml` file containing structured metadata for over 900 projects, including GitHub identifiers, package manager references, licenses, and category classifications. An automated GitHub Action workflow periodically fetches live metrics (stars, downloads) from GitHub and package registries, recalculates a quality score, and regenerates the `README.md` and changelog files in the `history/` directory. The repository serves as a comprehensive, living directory of the Python ML landscape, tracking trends, identifying dead or inactive projects, and providing a structured, versioned dataset for ecosystem analysis. While it produces a human-readable README, its true value lies in the structured data and the automated curation pipeline, making it a hybrid of a data repository and a content generation system.

RATING: 8.4/10 REPRO: 7.0/10 MAINT: active DRIFT: low
â–¸ 3 reusable components found
timqian/chinese-independent-blogs ★ 23845 2692 DATA_RESEARCH

This repository is a curated, community-maintained list of Chinese independent blogs. The primary data asset is a CSV file (`blogs-original.csv`) containing blog names, URLs, RSS feed URLs, and tags. The repository includes a set of Python utility scripts (`linter.py`, `readme_render.py`, `opml_generator.py`) that validate the CSV data, generate the README markdown, and produce an OPML feed export. These are orchestrated via GitHub Actions on push and weekly cron. The project is not a software application or a data pipeline in the traditional sense; it is a data curation project with tightly coupled orchestration scripts. The value lies in the curated dataset and the reproducibility of the derived outputs (README and OPML). The codebase is simple but well-engineered for its purpose: validation is thorough, transformations are idempotent, and CI ensures data integrity. This repository serves as a canonical reference for Chinese independent blogs and can be reused by other curation projects looking for a structured CSV-to-markdown/OPML workflow.

RATING: 8.2/10 REPRO: 9.0/10 MAINT: maintained DRIFT: low
â–¸ 3 reusable components found
brave-people/Dev-Event ★ 8855 631 DATA_RESEARCH

This repository is a community-curated, chronological database of Korean developer events (conferences, meetups, hackathons, bootcamps) spanning from 2020 to 2026. The core data asset is a collection of markdown files organized by year and month (`end_event/`), each containing structured event entries with titles, links, categories, organizers, and dates. It also archives event replays (`replay_event/`) and reviews (`review/`). The repository functions as the canonical data source for a broader ecosystem of web and mobile applications, but contains no application code itself. Its primary value lies in its longitudinal coverage of the Korean tech event landscape and its successful community-driven curation model. The data is explicitly protected by a proprietary license and a restrictive `robots.txt` that blocks AI/ML crawlers, signaling a strong stance on data governance. While the data model is simple and manually maintained, leading to some inconsistencies and link rot, the project's longevity and active community make it a valuable resource for understanding the Korean developer ecosystem.

RATING: 8.2/10 REPRO: 5.0/10 MAINT: active DRIFT: low
â–¸ 3 reusable components found
apple/password-manager-resources ★ 4803 636 DATA_RESEARCH

This repository is a curated collection of website-specific 'quirks' (hard-coded behaviors) for password managers, maintained by Apple. It contains structured JSON datasets including password rules (to generate compatible passwords), shared credential relationships (domains that share login backends), change-password URLs, 2FA appending sites, and web browser extension distribution information. The repo also includes reference implementations: a Password Rules language parser (JavaScript) and a regex-to-Password-Rules converter, along with validation and formatting scripts. While primarily a data curation project, the tools are reusable components for any password manager implementation. The data is validated against JSON schemas and linted in CI, ensuring high quality. The repository is actively maintained with frequent contributions from the community.

RATING: 8.2/10 REPRO: 8.0/10 MAINT: active DRIFT: low
â–¸ 2 reusable components found
helium/denylist ★ 199 212 DATA_RESEARCH

This repository maintains a denylist of Helium Hotspot public keys used to exclude specific Hotspots from Proof-of-Coverage challenges. It provides both human-readable CSV lists and machine-readable binary xor filters. The repository features a multisig approval process for updates, automated CI/CD pipeline for generating signed artifacts, and clear contribution guidelines. While the core data is simple lists of public keys, the release process demonstrates solid data engineering practices for secure, versioned data distribution.

RATING: 8.2/10 REPRO: 8.0/10 MAINT: active DRIFT: low
â–¸ 2 reusable components found
nextlevelbuilder/ui-ux-pro-max-skill ★ 132371 14040 APPLICATION

This repository provides a comprehensive AI-assisted UI/UX design system toolkit, primarily packaged as a skill for AI coding assistants (Claude Code, Cursor, Windsurf, Copilot, etc.) and distributed via the 'ui-ux-pro-max-cli' npm package. Its core functionality includes a curated knowledge base of 79 UI styles, 192 product color palettes, 74 font pairings, 119 UX guidelines, 22 tech stacks, and 192 reasoning rules – all stored as structured CSV files. A BM25 search engine (implemented in Python) enables domain‑aware retrieval, while a design system generator aggregates search results and applies reasoning rules to output complete design system recommendations (colors, typography, style, layout patterns, motion, anti‑patterns). The gallery Next.js app provides a browsable, filterable interface for the style catalog. Architecturally, the project is a hybrid of data curation and application delivery. The canonical source of truth lives in `src/ui-ux-pro-max/`, which is synchronised to `cli/assets/` and `.claude/skills/` via a CI‑enforced sync script. It includes sub‑skills for brand management, slide/presentation generation, banner design, icon/logo generation, and UI styling (with shadcn/ui and Tailwind integration). The CLI tool installs these skills into the appropriate AI assistant configuration directories, with platform‑specific templates for 20+ assistants. Automated workflows refresh Google Fonts and Phosphor icon catalogs weekly, and a suite of Python tests validates data contracts, stack freshness, and search relevance. Despite its active maintenance and rich functionality, the codebase suffers from significant file duplication across three directory trees, hardcoded version pins for frameworks, and test brittleness tied to exact data row counts. The design system generator and search engine are well‑crafted and reusable, but the overall project is tightly coupled to its curated CSV schema and the ecosystem of AI coding assistants. It represents a mature, production‑grade toolkit for developers who want AI‑assisted design generation, but the heavy duplication and external API dependencies (Gemini, Atlas Cloud, MuAPI) introduce moderate technical drift risk.

RATING: 8.0/10 MAINT: active DRIFT: medium
â–¸ 5 reusable components found
wy-luke/All-Chinese-Character-Set ★ 1946 215 DATA_RESEARCH

This repository provides curated Chinese character sets (3500 common, 7000 common, and a comprehensive set) combined with symbols (Latin letters, digits, punctuation, full-width variants). The data is sourced from official Chinese character frequency lists and community-maintained character tables. The repository includes simple Node.js scripts for building the final text files from source files, merging new characters, and removing duplicates. The primary use case is generating character sets for Unity TextMeshPro font creation, but the data can be used in any application requiring a comprehensive Chinese character inventory. The codebase is minimal and well-documented in AGENTS.md, with clear instructions for editing source files and regenerating outputs. The repository is actively maintained with no open issues and a recent commit.

RATING: 7.8/10 REPRO: 8.0/10 MAINT: active DRIFT: low
â–¸ 2 reusable components found
joylarkin/Awesome-AI-Market-Maps ★ 343 21 DATA_RESEARCH

This repository is a curated, structured list of 500+ AI startup market maps from 2024 to 2026, organized by quarter and subcategory (e.g., AI Agents, Compute, Healthcare). The primary artifact is a README.md with human-readable links, supplemented by a machine-readable CSV (`ai_market_maps.csv`), an RSS feed (`feeds/AIMarketMaps.xml`), and a Hugging Face dataset. The RSS feed is programmatically generated by `scripts/generate_rss.py`, which parses the README markdown and uses git history to assign publication dates. The repository serves founders, investors, and analysts by providing a comprehensive, frequently updated landscape of AI startup ecosystems. It is primarily a data curation and distribution project rather than a software application; the only code component is the RSS generation script and its associated CI workflow.

RATING: 7.7/10 REPRO: 6.0/10 MAINT: active DRIFT: low
â–¸ 1 reusable components found
zengin-code/source-data ★ 203 38 DATA_RESEARCH

This repository is the canonical source data for the ZenginCode ecosystem, providing a curated, machine-readable dataset of Japanese bank codes and branch codes. It contains structured branch information (code, name, kana, hira, roma) for thousands of financial institutions across Japan, stored in both JSON and YAML formats under data/branches/. The data is automatically updated daily via a GitHub Actions workflow that downloads the official Zengin bank branch list from an external source, converts it from Shift_JIS CSV to UTF-8, and generates per-bank branch files. The repository also includes a Ruby CLI tool (bin/zengin) that orchestrates the download, conversion, and commit process, making the pipeline reproducible. While primarily a data repository, it includes a well-structured ETL pipeline that can serve as a template for similar data curation projects. The dataset is consumed by language-specific packages (Ruby, JavaScript, Python) to provide bank branch lookup functionality in financial applications.

RATING: 7.7/10 REPRO: 8.0/10 MAINT: maintained DRIFT: low
â–¸ 3 reusable components found
jkwakman/Open-Cookie-Database ★ 314 117 DATA_RESEARCH

The Open Cookie Database is a community-curated reference dataset cataloging HTTP cookies. The single source of truth is open-cookie-database.csv, which records each cookie's platform, category (Functional, Personalization, Analytics, Marketing, Security), cookie name/pattern, domain, description, retention period, data controller, GDPR/privacy policy URL, and a wildcard-match flag. The dataset is maintained through manual PR-based contributions following a documented research workflow, and it is consumed by downstream privacy tooling (notably the EU EDPB Website Auditing Tool, via a dedicated EDPB JSON export). A static HTML viewer renders the CSV client-side using PapaParse and DataTables for browsing and export. The data lineage is simple and fully automated: on every push touching the CSV, GitHub Actions runs validate.py (UUID format/uniqueness, category whitelist, duplicate detection), then regenerates two derived artifacts - open-cookie-database.json (records grouped by platform) via convert_csv_to_json.py, and open-cookie-database-edpb.json via convert_edpb.py, which maps the flat CSV onto the EDPB tool's nested knowledge-base JSON schema. The pipeline is deterministic, idempotent, and commits the regenerated JSON back to the repo when changed. There is no application runtime, API, or library; the three Python scripts and CI workflows are infrastructure that exists solely to keep the dataset valid and in sync across formats. Craftsmanship is functional but modest. The scripts show reasonable engineering discipline: CLI parameterization, graceful handling of missing/empty files, GitHub Actions error annotations, and non-zero exit codes on failure. The validation script encodes real data-quality rules. However, there are clear signs of aging: the HTML viewer references the retired StackPath Bootstrap CDN and contains a malformed protocol-relative script URL ('//http://...'); the main validation workflow still pins Node 16 (EOL) and uses checkout@v2; convert_edpb.py uses the deprecated datetime.utcnow(); and pandas is installed unpinned. These issues do not break the core CSV-to-JSON pipeline, but they make the supporting surface fragile. The repository is a data asset first and code second - the value lies in the curated cookie catalog and the governance around it, not in the software. It is a solid, genuinely useful open-data project with competent but unremarkable data engineering. An analyst can clone it and reproduce all derived JSON artifacts with just Python and pandas, since the dataset is bundled and the transforms are simple, explicit, and parameterized.

RATING: 7.1/10 REPRO: 7.5/10 MAINT: maintained DRIFT: medium
â–¸ 3 reusable components found
klsdf/GreenResourcesManager ★ 1024 42 APPLICATION

GreenResourcesManager is an Electron + Vue 3 desktop application for local-first management of personal digital media collections—games, images, videos, novels, audio, websites, and software. It provides a config-driven UI with customizable pages, tag-based filtering, batch import, and built-in readers (EPUB, PDF, comic viewer, video player). Notable features include a desktop pet gamification system (affection, hunger, earnings), a disguise mode for privacy, a plugin system, and a local HTTP API for external access. Data is stored in SQLite with legacy JSON file backup. The application is actively developed (last commit April 2026) with a sizable feature set and internationalization (zh-CN, zh-TW, en, ja). However, the codebase shows significant technical drift: near-duplicate main-process code in `electron/` and `public/js/`, incomplete migration from JSON to SQLite, hardcoded Windows-specific paths, and extensive debug logging scattered throughout. The pet subsystem is tightly coupled and the plugin system remains marked as under development. While the config-driven UI pattern and composable architecture offer reuse potential, overall craftsmanship is average—functional but with notable boilerplate and coupling.

RATING: 6.8/10 MAINT: active DRIFT: medium
â–¸ 4 reusable components found
vlongle/pixie ★ 303 18 APPLICATION

This repository implements Pixie, a research framework for predicting dense physical material fields (density, Young's modulus, Poisson's ratio, material ID) from 2D image observations of 3D objects. Hydra configs drive a multi-stage pipeline: Objaverse object retrieval, Blender rendering, NeRF-based CLIP feature field training (F3RM), Gaussian Splatting training, voxel grid feature extraction, VLM-based or neural material segmentation, and MPM physics simulation via the vendored PhysGaussian codebase. The pipeline.py entry point orchestrates these stages with caching and reproducibility checks, while render.py produces Blender visualizations of segmented physics fields. Real-scene mode extends the same pipeline to captured datasets using COLMAP dataparsers. The repository also contains the full PixieVerse data curation workflow: class-based Objaverse selection using sentence-transformers, large-scale parallel Blender rendering, VLM-based filtering and critique (Gemini/Claude/GPT/Qwen via the vendored vlmx agent framework), and Streamlit tools for human-in-the-loop label correction and simulation validation. The discrete (semantic segmentation) and continuous (property regression) 3D UNet training lives in the vendored Wavelet-Generation submodule, with shared utilities in pixie/training_utils.py and normalization statistics already computed in normalization_stats/. Architecturally this is a hybrid research/application repository: primarily a configuration-driven pipeline framework with CLI entry points, but also a dataset-generation and model-training workflow. It depends heavily on vendored third-party code (nerfstudio, f3rm, PhysGaussian, Wavelet-Generation, vlmx) and shells out to Blender, CUDA rasterizers, and Hugging Face. Quality is research-grade: disciplined Hydra configs, contextual config snapshots, and skip-if-exists caching coexist with hardcoded user-specific paths, commented-out dead code, and a subprocess-heavy orchestration style. Download scripts for pre-trained models and the PixieVerse dataset are included and functional.

RATING: 6.8/10 MAINT: maintained DRIFT: medium
â–¸ 5 reusable components found
── end of results ──
$ submit: