Made With ML is an educational data science project that implements an end-to-end ML pipeline for classifying GitHub repositories into categories (NLP, computer vision, MLOps, other) based on their titles and descriptions. The core data lineage flows from raw CSV datasets (dataset.csv, holdout.csv) through text cleaning, tokenization via SciBERT, and stratified splitting into train/val/test sets, followed by distributed training with PyTorch and Ray Train, hyperparameter tuning via HyperOpt, evaluation with slice-based metrics using Snorkel, and serving through a FastAPI/Ray Serve deployment. The project demonstrates MLOps patterns including MLflow experiment tracking, data validation with Great Expectations, behavioral model testing, and CI/CD via GitHub Actions with Anyscale for cloud execution. The codebase is well-structured with clear separation of concerns across data.py, train.py, tune.py, evaluate.py, predict.py, serve.py, models.py, and utils.py. It includes comprehensive tests at the code, data, and model levels. However, the project shows signs of staleness with pinned dependencies from 2023 (Ray 2.7.0, transformers 4.28.1, torch 2.0.0) that have known security vulnerabilities and compatibility issues with current Python versions. The data pipeline components—particularly the CustomPreprocessor, stratified splitting, and text cleaning—are genuinely reusable patterns. The evaluation framework with slice-based metrics and behavioral tests (invariance, directional expectations, minimum functionality) represents solid ML engineering practice. The deployment configuration is tightly coupled to Anyscale's proprietary platform, limiting portability.
This repository is a curated skill library for data analysis workflows, designed to be consumed by AI agents (Claude Code, Codex). It contains 31 skills organized into 6 categories: data quality validation, documentation/knowledge management, data analysis/investigation, data storytelling/visualization, stakeholder communication, and workflow optimization. Each skill is defined by a SKILL.md file with YAML frontmatter, a step-by-step process, required inputs/outputs, and optional scripts, reference materials, and asset templates. The scripts are standalone Python CLIs that generate synthetic data for demo purposes and can be adapted to real datasets. The repo is not a deployable application or a data pipeline; it is a structured knowledge base and toolset for performing data analysis tasks programmatically. It covers common analytical patterns such as null counting, duplicate detection, cohort retention, funnel conversion, A/B testing, time series anomaly detection, metric calculation, and more. The codebase is well-organized and follows consistent conventions, but shows signs of staleness (future dates in README and LICENSE, no recent activity). Dependencies are not pinned, and no environment definitions are provided, limiting reproducibility without manual setup.