MLJ.jl is the umbrella entry point and coordination layer for the MLJ machine-learning ecosystem in Julia. The repository's own source is a thin facade module (`src/MLJ.jl`) that re-exports the public API from satellite packages — MLJBase, MLJModels, MLJTuning, MLJEnsembles, MLJIteration, and StatisticalMeasures — plus a small amount of scitype-related code. The actual implementations of fit/predict/transform, learning networks, tuning, ensembling, model composition, and evaluation live primarily in those dependencies. This repo is therefore best understood as the distribution, documentation, integration-testing, and governance hub for the larger MLJ framework rather than a standalone implementation of ML algorithms. In production, users enter through `using MLJ`, load concrete models via `@load`, wrap data and models in `machine(...)`, and drive the system with `fit!`, `predict`, `transform`, `evaluate`, `TunedModel`, `IteratedModel`, `Pipeline`, and related APIs. The system is designed around scientific types and the Tables.jl interface, allowing tabular data to flow uniformly into 200+ models across Julia and other languages, including scikit-learn interop. It supports complex composite workflows: linear pipelines, learning networks/DAGs, target transformations, homogeneous ensembles, stacking, hyperparameter optimization, and performance evaluation. The repository also contains an extensive documentation site, runnable examples, a JOSS paper, and a registry-driven integration test suite. Maintenance is active: the latest commit is recent, CI runs on GitHub Actions, dependabot is configured, and release automation via TagBot is present. Some documentation artifacts show age — for example, the synthetic-data notebook records Julia 1.4.2 — and a few APIs are explicitly experimental, but the core package remains current with the Julia ecosystem and demonstrates disciplined package management across a large dependency graph.
ML.NET is a cross-platform, open-source machine learning framework for .NET developers. It provides a comprehensive set of tools for building, training, and deploying custom machine learning models without requiring expertise in Python or R. The framework covers traditional ML algorithms (classification, regression, clustering, ranking, time series), deep learning via TensorFlow and ONNX integration, automated machine learning (AutoML) with hyperparameter sweeping and pipeline search, and generative AI (GenAI) support for large language models like LLaMA, Phi, and Mistral. It also includes a DataFrame API for data analysis and a code generator for producing prediction code. The architecture is modular, with core abstractions like IDataView, IEstimator, and ITransformer forming a composable pipeline pattern. The repository is actively maintained by Microsoft and has a large community.
traffic is a Python library and CLI tool for processing, analyzing, and visualizing air traffic data. It ingests trajectory data from multiple sources including OpenSky Network (via REST/Trino), FlightRadar24, Eurocontrol (DDR, AIXM), FAA, and local ADS-B decoders. The core abstractions are `Flight` (single trajectory) and `Traffic` (collection) built on pandas DataFrames, with extensive methods for cleaning, resampling, filtering, geometric operations, navigation event detection (landing, go-around, holding patterns), performance estimation (fuel flow, emissions), and clustering. The library provides a declarative lazy evaluation pipeline (`LazyTraffic`) that chains operations before a single `.eval()` call, enabling efficient parallel processing. It also offers visualization backends for Matplotlib, Altair, Leaflet, and Plotly, and includes a Vue.js/Deck.gl plugin (Tangram) for real-time landing alignment in a web dashboard.
PyCaret 4.0 is a full-stack MLOps platform combining a low-code Python AutoML engine, a FastAPI control plane server, and a React web UI into a single deployable system. The engine provides OOP Experiment classes for classification, regression, clustering, anomaly detection, and time series, with native sklearn pipeline implementations and typed result dataclasses. The server adds multi-user workspace management, experiment tracking, model registry, deployment serving, drift monitoring, LLM-powered advisory copilots, scheduling, webhooks, and audit logging. The React frontend offers a rich SPA with real-time event streaming, interactive plots (Plotly), and data-driven forms generated from engine introspection. The architecture is well-structured with protocol-based abstractions for storage (local-fs/S3), data sources (CSV/Postgres), LLM providers (Anthropic/OpenAI/fake), and notebook backends. This is primarily an application (ML platform) but has strong data/research characteristics due to the engine's statistical analyses, dataset profiling, and time-series utilities. The codebase is under active development with a systematic migration from legacy functional APIs to the new OOP architecture, and many components are genuinely reusable.
This repository is a comprehensive educational curriculum for Machine Learning, structured as a collection of Jupyter notebooks, data files, and instructional documentation. While it contains small application examples (a Vue.js quiz app and a Flask web app demo), the dominant character is data research and education, focusing on teaching ML concepts through practical data science workflows. The content covers Regression, Classification, Clustering, NLP, Time Series, and Reinforcement Learning, utilizing Scikit-learn as the primary modeling library. The codebase is organized into modular lesson directories, each containing notebooks, assignments, and solutions. It demonstrates a strong commitment to accessibility with extensive multi-language translations and a devcontainer setup for consistent environments. However, reproducibility is impacted by a reliance on external datasets (Kaggle, Dropbox) which are prone to link rot, and scattered dependency management across different lesson folders rather than a unified root configuration. Technically, the repository shows signs of medium drift, particularly in the quiz application which utilizes Vue 2 patterns instead of the modern Vue 3 Composition API. The core ML content remains relevant, but the educational infrastructure (translation services, external links) requires ongoing maintenance to ensure accuracy and availability. Despite this, it stands as a high-quality, reusable resource for learning ML pipelines and data engineering patterns.
This repository is a comprehensive educational curriculum for Data Science, structured as a 10-week, 20-lesson course designed for beginners. It functions primarily as a data_research asset, containing Jupyter notebooks, R scripts, Python examples, and curated datasets (CSV, TSV) used for Exploratory Data Analysis (EDA) and visualization. The content covers the full data lifecycle, from data definition and ethics to cloud deployment on Azure, utilizing tools like Pandas, Matplotlib, Seaborn, and ggplot2. While the core is educational data content, it includes a hybrid application component: a Vue.js-based quiz application (`quiz-app`) used for assessments. The repository is heavily localized, with content translated into dozens of languages using an AI translation service, which introduces a specific maintenance pattern involving disclaimers about translation accuracy. The data lineage is well-documented, sourcing datasets from Kaggle, USDA, and Wikipedia, with clear licensing information provided. The codebase is optimized for learning rather than production engineering. Notebooks serve as the primary execution environment, demonstrating data cleaning, visualization, and basic statistical analysis. The architecture is flat and modular, organized by lesson topics (Introduction, Working With Data, Visualization, Lifecycle, Cloud). It is highly valuable for educators and learners seeking reproducible data science examples, though the heavy reliance on AI translation and mixed Vue versions (2 vs 3) presents technical drift risks.
This repository provides a build system for version-stable Rocker Docker images for R, including r-ver, rstudio, tidyverse, verse, shiny, shiny-verse, and geospatial. The core architecture uses R scripts to generate Dockerfiles from templates and docker-bake.json files from version matrices, enabling reproducible builds with fixed CRAN snapshots, RStudio Server versions, and CTAN dates. The CI/CD pipeline (GitHub Actions) automatically updates container definitions, builds multi-platform images (linux/amd64 and linux/arm64), and publishes them to Docker Hub and GitHub Container Registry. The repository also contains a collection of standalone installation scripts (rocker_scripts) that can be reused in downstream Dockerfiles to install R, RStudio Server, Shiny Server, tidyverse packages, geospatial libraries, Python, Julia, Jupyter, and more. These scripts are parameterized, idempotent, and follow best practices for Docker image layering. The build system itself is a well-engineered example of code generation and matrix-based CI, though it is tightly coupled to the Rocker project's specific versioning scheme.
This repository is a curated, ranked list of open-source Python machine learning libraries, known as 'Best-of Machine Learning with Python'. It functions primarily as a data curation and ecosystem analysis project. The core artifact is a `projects.yaml` file containing structured metadata for over 900 projects, including GitHub identifiers, package manager references, licenses, and category classifications. An automated GitHub Action workflow periodically fetches live metrics (stars, downloads) from GitHub and package registries, recalculates a quality score, and regenerates the `README.md` and changelog files in the `history/` directory. The repository serves as a comprehensive, living directory of the Python ML landscape, tracking trends, identifying dead or inactive projects, and providing a structured, versioned dataset for ecosystem analysis. While it produces a human-readable README, its true value lies in the structured data and the automated curation pipeline, making it a hybrid of a data repository and a content generation system.
DoWhy is a mature Python library for causal inference that implements a systematic four-step workflow: model (encode causal graph), identify (derive estimand), estimate (compute causal effect), and refute (validate robustness). The library supports two major frameworks: (1) classical potential-outcomes-based estimation with ATE/ATT/CATE via backdoor, frontdoor, instrumental variables, and mediation analysis, leveraging multiple estimators including linear regression, propensity score methods, doubly robust, instrumental variable, regression discontinuity, and integrations with EconML and CausalML; (2) Graphical Causal Model (GCM) based inference for what-if analysis, counterfactual sampling, distribution change attribution, arrow strength, intrinsic causal influence, anomaly attribution, and root cause analysis. The architecture is modular with abstract base classes for estimators, refuters, interpreters, and GCM components. The main entry point is the CausalModel class, which orchestrates identification, estimation, and refutation via a consistent API. Key dependencies include scikit-learn, statsmodels, networkx, sympy, pandas, numpy, and optional integrations with Torch, EconML, Causal-learn, and TabPFN. The library is production-ready with comprehensive test suites, CI/CD, and extensive documentation, and is actively maintained with a large contributor base.
Plots.jl is a comprehensive, multi-backend plotting library for Julia, organized as a monorepo containing several interdependent packages: PlotsBase (core plotting infrastructure), RecipesBase/RecipesPipeline (a macro-driven recipe system for defining custom plot types), GraphRecipes (graph and tree visualization), StatsPlots (statistical plots like boxplots, violin plots, histograms, and correlation plots), and PlotThemes (color themes). The library provides a unified API over backends including GR, PythonPlot, PlotlyJS, PGFPlotsX, UnicodePlots, and Gaston, allowing users to switch between interactive and static output without changing code. The architecture is centered on a preprocessing pipeline that transforms input data and attributes through alias resolution, magic argument expansion, and recursive recipe application before delegating to the chosen backend. This design enables extensibility: package authors can define new plot types via the lightweight RecipesBase dependency without pulling in the full Plots stack. While primarily a general-purpose visualization library, it includes strong support for scientific and research workflows through statistical plots, graph layouts, unitful quantities, HDF5 persistence, and animations. The codebase is large and macro-heavy, with some backends lagging in feature parity and occasional deprecated API usage, but overall it is actively maintained and well-tested.
Cookiecutter Data Science (CCDS) is a CLI tool that generates standardized, opinionated project scaffolds for data science work. It wraps the cookiecutter templating engine with custom prompting logic that supports nested configuration choices (e.g., cloud storage options with sub-fields for bucket names), and post-generation hooks that dynamically write dependency files in multiple formats (requirements.txt, pyproject.toml, environment.yml, Pipfile, pixi.toml), configure linting/formatting tools (ruff or flake8+black+isort), set up testing frameworks (pytest/unittest), and scaffold code modules with boilerplate for data processing, feature engineering, modeling, and visualization. The tool is designed to be run via the `ccds` command, which defaults to the DrivenData template repository and the installed version of the template. The codebase demonstrates solid software engineering: it monkey-patches cookiecutter's internals in a controlled way to add functionality (prompting with sub-options, custom context file selection), provides a suite of reusable hook utilities for dependency management and custom config injection, and maintains extensive test coverage across multiple operating systems and environment managers (conda, virtualenv, pipenv, uv, pixi, poetry). The architecture is a thin CLI application that delegates to cookiecutter while extending it through monkey-patching and post-generation hooks. It is not a data research project itself but a tool for creating such projects.
tsfresh is a Python library for automatic time series feature extraction and selection, grounded in the FRESH (FeatuRe Extraction based on Scalable Hypothesis tests) algorithm. It provides over 700 feature calculators spanning statistics, signal processing, and nonlinear dynamics, along with hypothesis-testing-based feature selection using Benjamini-Hochberg or Benjamini-Yekutieli FDR control. The library is designed for integration into scikit-learn pipelines via transformers like FeatureAugmenter and RelevantFeatureAugmenter, and supports parallel/distributed execution via multiprocessing, Dask, or Spark. While primarily a software library (application), it has strong research roots with academic publications and example notebooks demonstrating classification, regression, and forecasting tasks. The codebase is well-structured with clean abstractions for data input normalization (adapters), pluggable parallelization (distributors), and feature settings reconstruction, making it a robust tool for time series feature engineering in production ML workflows.
StatsForecast is a production-grade Python library for large-scale univariate time series forecasting. It provides a unified API (fit/predict/forecast/cross_validation) over 40+ statistical models including AutoARIMA, AutoETS, AutoCES, AutoTheta, MSTL, TBATS, GARCH, and intermittent-demand methods (Croston, ADIDA, IMAPA). The core algorithms are implemented in C++ via pybind11 for performance, with Python wrappers handling model selection, prediction intervals, and conformal prediction. The library supports distributed execution on Spark, Dask, and Ray through a Fugue-based backend, and integrates with pandas, Polars, and MLFlow. While primarily an application library, the repository also contains extensive benchmarking experiments against R packages, Prophet, and NeuralProphet, as well as tutorials and competition notebooks, giving it a hybrid character. The codebase is well-structured with a clear separation of concerns (core API, model implementations, distributed backends, utilities) and includes a comprehensive test suite validated against R reference values.
This repository is a curated resource list for data science education and practice, following the popular 'awesome-' list format. It does not perform data research itself but aggregates links to tutorials, courses, tools, books, and other learning materials across the data science spectrum. The repository's primary function is reference and education rather than active data processing or analysis. While classified as data_research due to its subject matter, the repository is fundamentally a documentation and curation project. It contains minimal executable code, consisting mainly of the README.md resource list, standard community health files (CODE_OF_CONDUCT.md, CONTRIBUTING.md), and a simple CI workflow for link validation. The value lies in the comprehensiveness and organization of the curated links rather than in technical implementation. The project demonstrates maintenance awareness through its link-checking CI setup and clear contribution guidelines. Despite being primarily a static resource list, it shows good practices for documentation projects: automated link validation to prevent link rot, structured contribution processes, and clear licensing. Its enormous popularity (nearly 30k stars) indicates sustained community value as a starting point for data science learners.
Matplot++ is a C++ graphics library for data visualization that provides a MATLAB-like API for creating publication-quality plots. It supports a wide range of plot types including line, scatter, bar, histogram, surface, contour, polar, geographic, and 3D plots, as well as image display, graph visualization, and annotations. The library is designed for scientific computing and engineering applications, offering both free-standing function calls and an object-oriented interface with method chaining. It features a pluggable backend architecture with Gnuplot as the primary rendering backend (via pipe communication) and an experimental OpenGL backend. The library handles data transformation, automatic tick calculation, colormap interpolation, contour generation (adapted from Matplotlib), histogram binning with multiple algorithms, and geographic map rendering using bundled Natural Earth data. External dependencies include Gnuplot 5.2.6+ at runtime, with optional support for image processing libraries (CImg, JPEG, TIFF, PNG, OpenCV) and numerical libraries (FFTW, LAPACK, BLAS). The build system is CMake-based with comprehensive options for testing, examples, and documentation generation. Architecturally, the library is organized around a core figure-axes-object hierarchy: figure_type manages the canvas and backend, axes_type holds plot objects and axis configuration, and individual plot types (line, bars, histogram, etc.) inherit from axes_object and implement backend-specific rendering methods. Free-standing functions in freestanding/ forward calls to the current axes via a global figure registry. The backend interface abstracts rendering commands, allowing the same plot objects to work with different backends. The library includes a large set of examples (200+) that serve as both documentation and integration tests, and a documentation pipeline that auto-generates a gallery from example executables. Key entry points for users are the matplot.h header and the free-standing functions (plot, hist, surf, etc.). The library is consumed via CMake's find_package or as a subdirectory. It supports both interactive (reactive) and non-interactive (quiet) modes, and can save plots in multiple formats (SVG, EPS, PNG, PDF, etc.) through backend-specific commands.
The pm4py repository is a process mining library for Python, providing various algorithms for process discovery, conformance checking, and enhancement. It supports multiple process mining techniques, including inductive mining, alpha miner, and heuristics miner. The library is well-structured, with a clear organization and documentation. However, the large number of modules and algorithms might pose maintenance challenges.
pytorch-forecasting is a deep learning library for time series forecasting, providing implementations of state-of-the-art models such as Temporal Fusion Transformer, N-BEATS, N-HiTS, DeepAR, PatchTST, TiDE, TimeXer, and many others. Built on PyTorch and PyTorch Lightning, it offers a complete pipeline from data preprocessing (TimeSeriesDataSet, encoders, samplers) through model training (with GPU support, callbacks, hyperparameter tuning via optuna) to prediction and evaluation (point, quantile, and distributional metrics). The library supports multivariate forecasting, static and time-varying covariates, categorical features, and multiple targets. A major v2 refactoring is underway, introducing a new data module layer and simplified package interface, though the v2 API is explicitly marked as experimental and unstable. The codebase is well-tested with automated integration tests across all models and metrics, and employs a composition pattern for metrics (supporting weighting, aggregation, and distribution losses) and a layered architecture for models (base classes, package containers, building blocks).
Made With ML is an educational data science project that implements an end-to-end ML pipeline for classifying GitHub repositories into categories (NLP, computer vision, MLOps, other) based on their titles and descriptions. The core data lineage flows from raw CSV datasets (dataset.csv, holdout.csv) through text cleaning, tokenization via SciBERT, and stratified splitting into train/val/test sets, followed by distributed training with PyTorch and Ray Train, hyperparameter tuning via HyperOpt, evaluation with slice-based metrics using Snorkel, and serving through a FastAPI/Ray Serve deployment. The project demonstrates MLOps patterns including MLflow experiment tracking, data validation with Great Expectations, behavioral model testing, and CI/CD via GitHub Actions with Anyscale for cloud execution. The codebase is well-structured with clear separation of concerns across data.py, train.py, tune.py, evaluate.py, predict.py, serve.py, models.py, and utils.py. It includes comprehensive tests at the code, data, and model levels. However, the project shows signs of staleness with pinned dependencies from 2023 (Ray 2.7.0, transformers 4.28.1, torch 2.0.0) that have known security vulnerabilities and compatibility issues with current Python versions. The data pipeline components—particularly the CustomPreprocessor, stratified splitting, and text cleaning—are genuinely reusable patterns. The evaluation framework with slice-based metrics and behavioral tests (invariance, directional expectations, minimum functionality) represents solid ML engineering practice. The deployment configuration is tightly coupled to Anyscale's proprietary platform, limiting portability.
PyRanges (version 0, legacy) is a Python library for genomic interval operations, analogous to Bioconductor's GenomicRanges. It provides a comprehensive set of tools for manipulating, comparing, and analyzing genomic intervals and annotations. The library supports reading and writing common bioinformatics file formats (BED, GTF, GFF3, BAM, BigWig), and implements a wide range of operations including merging, intersection, subtraction, join, nearest neighbor, clustering, coverage, and statistical methods like Jaccard and Fisher exact. Internally, PyRanges represents intervals as a dictionary of pandas DataFrames keyed by chromosome and optionally strand. This design enables efficient per-chromosome parallelism and easy integration with the Python data science stack. Operations are implemented as modular functions in separate files under the `methods/` directory, each accepting a DataFrame and keyword arguments from a central dispatcher. The library also provides a parallel execution framework using Ray via `pyrange_apply` and related functions, allowing users to apply custom functions to each chromosome/strand group in parallel. Although primarily a software library, this repository has a strong research character: it includes built-in example datasets (e.g., ChIP-seq peaks, CpG islands, Ensembl annotations), a bookdown tutorial with executable R Markdown, property-based tests that compare results against the bedtools suite, and statistical methods designed for colocalization and annotation enrichment. This makes it equally valuable as a research tool for bioinformatics analysis. Maintenance note: This is the legacy version 0 of PyRanges, superseded by a redesigned version 1 (pyranges1). Despite the deprecation, the codebase is mature, well-tested, and still functional. The design patterns (modular operations, per-chromosome dispatch, property-based testing against a reference tool) are noteworthy and could be reused in other bioinformatics and data engineering projects.
This repository is a curated list of awesome Jupyter projects, libraries, and resources. It serves as a reference for developers and data scientists seeking tools to enhance their Jupyter experience, including runtimes, visualization libraries, collaboration tools, and more. The list is organized into thematic categories (e.g., Runtimes/Frontends, Collaboration/Education, Visualization) for easy navigation and discovery. The repository includes maintenance workflows to ensure the quality and consistency of the list. A GitHub Actions workflow runs awesome-lint and remark-lint to validate the format of the list and keep it alphabetized. Another workflow tracks GitHub traffic statistics. The project follows standard contribution guidelines and is licensed under Creative Commons Attribution-ShareAlike 4.0. While the repository itself does not contain executable code or data, it provides significant value as a discovery tool for the Jupyter ecosystem. The maintained workflows demonstrate good practices for keeping awesome lists clean and consistent, which can be reused in similar projects. The content is regularly updated, as evidenced by recent commits, indicating active maintenance.