This repository is a curated collection of website-specific 'quirks' (hard-coded behaviors) for password managers, maintained by Apple. It contains structured JSON datasets including password rules (to generate compatible passwords), shared credential relationships (domains that share login backends), change-password URLs, 2FA appending sites, and web browser extension distribution information. The repo also includes reference implementations: a Password Rules language parser (JavaScript) and a regex-to-Password-Rules converter, along with validation and formatting scripts. While primarily a data curation project, the tools are reusable components for any password manager implementation. The data is validated against JSON schemas and linted in CI, ensuring high quality. The repository is actively maintained with frequent contributions from the community.
This repository is an awesome list curating research papers on Video-LMM Post-Training for video reasoning. It maintains a JSON dataset of paper metadata (title, release date, links, tags, etc.) and uses a Python script to generate a categorized README.md with markdown tables organized by research paradigm (Reinforced Video-LMMs, SFT for Reasoning, Test-Time Scaling, Benchmarks, Surveys). The repository serves as a reference tracker for the latest advancements in enhancing reasoning capabilities of video-focused multimodal models through post-training techniques. The hybrid nature lies in its dual role as both a data collection (the JSON paper metadata) and a lightweight data presentation system (the generation script).
This repository is a curated survey of research papers on World Action Models (WAM), maintained as a living list in a single README.md file. It categorizes 109 papers into three families: Render-and-Decode, Latent-Only, and Video-Generation-Free, with each entry including publication date, title, backbone, substrate, and a one-line summary. The repository serves as a reference resource for researchers in robotics and embodied AI, providing a structured overview of recent advances in WAM methodologies. While it contains no executable code or data processing pipelines, it is actively updated via community contributions through pull requests.
This repository is a community-curated, chronological database of Korean developer events (conferences, meetups, hackathons, bootcamps) spanning from 2020 to 2026. The core data asset is a collection of markdown files organized by year and month (`end_event/`), each containing structured event entries with titles, links, categories, organizers, and dates. It also archives event replays (`replay_event/`) and reviews (`review/`). The repository functions as the canonical data source for a broader ecosystem of web and mobile applications, but contains no application code itself. Its primary value lies in its longitudinal coverage of the Korean tech event landscape and its successful community-driven curation model. The data is explicitly protected by a proprietary license and a restrictive `robots.txt` that blocks AI/ML crawlers, signaling a strong stance on data governance. While the data model is simple and manually maintained, leading to some inconsistencies and link rot, the project's longevity and active community make it a valuable resource for understanding the Korean developer ecosystem.
This repository is a curated, structured dataset of C and C++ standard features (from C99 to C++29) with per-compiler support status across multiple toolchains (GCC, Clang, MSVC, Xcode, etc.). It includes YAML data files defining features, toolchain releases, and frontend metadata, along with a C++ validation tool that enforces schema compliance and semantic consistency (e.g., verifying that referenced tool versions exist and issue trackers are properly used). The dataset is designed to power a reference website (cppstat) and is accompanied by detailed Markdown explanations for each feature. While the primary artifact is the data itself, the validator and CI pipeline demonstrate strong data engineering practices, making the repository a reliable source for compiler feature support information.
This repository maintains a denylist of Helium Hotspot public keys used to exclude specific Hotspots from Proof-of-Coverage challenges. It provides both human-readable CSV lists and machine-readable binary xor filters. The repository features a multisig approval process for updates, automated CI/CD pipeline for generating signed artifacts, and clear contribution guidelines. While the core data is simple lists of public keys, the release process demonstrates solid data engineering practices for secure, versioned data distribution.
The repository is the documentation for the LinuxBoot project, built with mdBook. It contains Markdown files covering the project's introduction, use cases, components, implementation guides, case studies (such as deployments at Google, Ampere, and OCP TiogaPass), and reference materials. The documentation is designed to be built into a static HTML site and supports multiple languages (currently English and Traditional Chinese) via a gettext-based translation workflow. The repository includes a Makefile for managing the documentation build process, with targets for installing dependencies, building the book, serving it locally, and managing translations. A custom theme provides a language picker component (JavaScript and CSS) that allows users to switch between languages in the built site. Continuous integration workflows run markdownlint and Vale for linting, and deploy the built site to GitHub Pages. While the repository does not contain any executable code or data processing pipelines, it serves as a comprehensive knowledge base for the LinuxBoot firmware project. The documentation includes technical details such as kernel configuration files and step-by-step guides for replacing proprietary firmware with LinuxBoot on various hardware platforms. The project is actively maintained, as evidenced by recent commits and ongoing issue activity. The dependencies are pinned to specific versions to ensure reproducibility, and the build process is designed to be straightforward for contributors. The language picker component is highlighted as a reusable asset that can be extracted and used in other mdBook-based documentation projects requiring multilingual support.
This repository serves as a comprehensive metadata catalog for NYC GIS datasets maintained by the Office of Technology and Innovation. It does not contain actual geospatial data or processing code, but rather detailed documentation describing over 30 NYC GIS datasets including 3D building models, address points, aerial imagery, building footprints, transportation networks, and environmental data. Each dataset is documented in a consistent markdown format covering identification (purpose, source, update frequency), data quality specifications (coordinate system, resolution, accuracy), and attribute information (for vector datasets). The repository functions as a reference for understanding what GIS data is available from NYC, its characteristics, and how to access it.
WebHackersWeapons is a curated, YAML-driven knowledge base and static documentation generator for web hacking and bug bounty tools, rather than a runnable application. Its core truth layer is 530+ weapons/*.yaml files, each defining a tool's name, URL, description, category, type, platform, language, and tags. These files are transformed by Ruby ERB scripts into a generated README and categorized pages by language and vulnerability tag, with schema validation enforced by scripts/validate_weapons.rb and CI via GitHub Actions. The repository behaves as a structured dataset with a small ETL pipeline: YAML metadata -> validated catalog -> Markdown cross-reference pages.
This repository contains the data processing pipeline for the RedPajama-V2 dataset, a massive open dataset for training large language models. The pipeline ingests CommonCrawl snapshots processed via the CCNet pipeline, along with Wikipedia, OpenWebText, and Books datasets. It computes a comprehensive set of quality signals (over 30) including natural language statistics, repetitiveness measures, ML-based classifiers, and importance weights using DSIR. Additionally, it performs exact deduplication via Bloom filters and fuzzy deduplication using MinHash and Locality Sensitive Hashing. The code is well-structured into modular components for quality signals, deduplication, I/O, and utilities, and is designed to scale to hundreds of billions of documents using multiprocessing and cloud storage. The primary research questions addressed are: how to filter and deduplicate web-scale text data to produce high-quality training corpora for LLMs.
This repository is a curated, community-maintained list of Chinese independent blogs. The primary data asset is a CSV file (`blogs-original.csv`) containing blog names, URLs, RSS feed URLs, and tags. The repository includes a set of Python utility scripts (`linter.py`, `readme_render.py`, `opml_generator.py`) that validate the CSV data, generate the README markdown, and produce an OPML feed export. These are orchestrated via GitHub Actions on push and weekly cron. The project is not a software application or a data pipeline in the traditional sense; it is a data curation project with tightly coupled orchestration scripts. The value lies in the curated dataset and the reproducibility of the derived outputs (README and OPML). The codebase is simple but well-engineered for its purpose: validation is thorough, transformations are idempotent, and CI ensures data integrity. This repository serves as a canonical reference for Chinese independent blogs and can be reused by other curation projects looking for a structured CSV-to-markdown/OPML workflow.
This repository provides a comprehensive baseline for person/object re-identification (ReID) using PyTorch. It implements a standard pipeline: data preparation (reorganizing images by identity), training with various backbones (ResNet, DenseNet, Swin, HRNet, etc.) and losses (cross-entropy, triplet, circle, contrastive, etc.), feature extraction, and evaluation (Rank-k, mAP). It also includes re-ranking (CPU and GPU-accelerated via custom CUDA kernels) and adversarial training. The code is well-structured and modular, with many self-contained loss functions and augmentation modules that can be reused in other metric learning projects. However, it is research-oriented: paths are hardcoded, dependencies are not fully pinned, and there is no containerized environment. The repository is actively maintained with support for modern architectures and PyTorch features.
OpenMoji is an open-source emoji set maintained by HfG Schwäbisch Gmünd. The repository contains the full pipeline for producing emoji assets: source SVGs, a metadata generation system that merges emojibase-data with custom CSV enhancements and Unicode draft overrides, export scripts for color/black SVGs, PNGs, PDFs, and font generation via Docker. The central data file `data/openmoji.json` provides structured emoji metadata including hexcodes, tags, authors, and dates. The build system is modular with incremental caching, color palette enforcement using kd-tree, and comprehensive tests for data integrity, SVG layer rules, and font table presence. While primarily a data/research repository producing emoji assets, it includes a thin npm package for programmatic access and a sophisticated build pipeline that could be adapted for other icon sets. The repository is actively maintained with recent commits and CI workflows, though it relies on external dependencies like emojibase-data and a Docker font builder that may introduce drift.
This is the DataTalksClub Data Engineering Zoomcamp — a free, instructor-led 9-week educational course repository, not a deployable application. It is a hybrid collection of teaching materials: Jupyter notebooks, dlt pipeline scripts, dbt models with tests, Airflow/Kestra/Spark/streaming examples, and workshop content. The dominant character is data_research: the repository exists to teach end-to-end data engineering through hands-on pipelines over NYC taxi trip data (yellow/green/FHV in CSV/Parquet from GCS) and the Open Library API. Data flows from raw sources through dlt extraction/normalization/loading into DuckDB/BigQuery, then through dbt transformations (staging/core layers) with singular/generic/unit tests and model contracts, plus batch processing with Spark and streaming with Kafka. The course is actively maintained with cohorts from 2022 through 2027, and includes a systematic image-distrust audit that removed 64 AI-generated illustrations from lessons.
Masader is a metadata catalog for Arabic NLP datasets, providing structured information about over 100 datasets including their tasks, dialects, sources, volumes, and licensing. The repository combines a Jekyll-based static site with a modular JavaScript frontend to enable dataset discovery through search, filtering, statistics visualization, and a natural language chatbot interface. While the core asset is the dataset metadata (JSON files in /datasets/), the web interface and associated tools (annotation, migration, validation scripts) form an application layer that makes the catalog usable and maintainable. The system supports dataset exploration via faceted search, demographic charts, and embedding-based clustering, with a chatbot for conversational queries. Metadata is kept up-to-date through migration scripts that handle schema evolution and validation checks that ensure consistency with external sources like venue links and volume totals.
This repository provides the data pipeline for the Sekai dataset, a large-scale egocentric video dataset for world exploration and generation research. It contains two main toolchains: dataset_downloading (extracting YouTube URLs from CSV annotations and downloading videos via yt-dlp) and clip_extracting (a multi-step GPU-accelerated pipeline to split raw videos into annotated clips, transcode them to H.265, and remux with audio). The pipeline is designed for the Sekai-Real subset and requires NVIDIA GPUs with NVENC support, CUDA 11.8, and specific Python dependencies. The code is well-structured with clear separation of steps, logging, and multiprocessing support. While tightly coupled to the Sekai dataset and hardware requirements, the utility modules for GPU video decoding/encoding and frame sampling are reusable in other video processing projects.
This repository is a curated reference list of academic papers and projects focused on style transfer using diffusion models. It organizes resources into categories including image synthesis (text-driven, image-driven, LoRA-based methods), video synthesis, 3D generation, style datasets, and related surveys. The repository contains no executable code, data, or training scripts; it serves solely as a bibliographic guide for researchers. Content is presented in markdown format with tables linking to arXiv papers and GitHub repositories, supplemented by badges for stars and publication dates.
This repository provides the official code, datasets, and checkpoints for the Timer family of large time series foundation models (Timer, Timer-XL, Sundial) from Tsinghua University. It implements a decoder-only Transformer architecture pre-trained on the Unified Time Series Dataset (UTSD) using the single-series sequence (S3) format. The code supports fine-tuning and evaluation on three downstream tasks: forecasting, segment-level imputation, and predictive anomaly detection. Data loading modules handle multiple time series formats (CSV, NPZ, TXT) with flexible windowing and scaling. The repository includes shell scripts for reproducible experiments across multiple datasets and data scarcity ratios. It also provides HuggingFace integration for zero-shot forecasting without training. While primarily a research codebase rather than a production application, it is well-structured with clean module separation and reusable components.
This repository is a data pipeline that transforms crowdsourced survey responses about Chinese university quality of life into a structured MkDocs static site. The core ETL script (`questionnaires/main.py`) ingests a CSV of desensitized survey results, applies name normalization via alias mapping, blacklist/whitelist filtering, and splits data into current and archived sets based on a cutoff date. It generates per-university Markdown files with Q&A sections, a navigation structure, and a MkDocs configuration. The pipeline is triggered by GitHub Actions on pushes to master, pushing generated content to a separate branch and to the production website repository. The data is licensed under CC BY-NC-SA 4.0. While the code is functional and well-organized for its purpose, it is tightly coupled to the specific survey schema and data files, limiting direct reuse as a library. The most reusable component is the `FilenameMap` class for generating unique slugs from entity names.
This repository is a curated resource list for data science education and practice, following the popular 'awesome-' list format. It does not perform data research itself but aggregates links to tutorials, courses, tools, books, and other learning materials across the data science spectrum. The repository's primary function is reference and education rather than active data processing or analysis. While classified as data_research due to its subject matter, the repository is fundamentally a documentation and curation project. It contains minimal executable code, consisting mainly of the README.md resource list, standard community health files (CODE_OF_CONDUCT.md, CONTRIBUTING.md), and a simple CI workflow for link validation. The value lies in the comprehensiveness and organization of the curated links rather than in technical implementation. The project demonstrates maintenance awareness through its link-checking CI setup and clear contribution guidelines. Despite being primarily a static resource list, it shows good practices for documentation projects: automated link validation to prevent link rot, structured contribution processes, and clear licensing. Its enormous popularity (nearly 30k stars) indicates sustained community value as a starting point for data science learners.