$ show scikit-learn/scikit-learn
scikit-learn is a production-grade machine learning library implemented in Python with Cython extensions for performance. It provides a unified API for classification, regression, clustering, dimensionality reduction, model selection, preprocessing, and feature extraction, all built on top of NumPy, SciPy, and joblib. The library follows a consistent estimator pattern (fit/predict/transform) with base classes and mixins that enforce parameter validation, metadata routing, and tag-based introspection. The architecture is modular, with each algorithm group (e.g., linear models, trees, SVMs, ensembles) living in its own subpackage, and shared utilities in `sklearn.utils`. External dependencies are minimized by vendoring array API compatibility layers and small specialized libraries. The repository is a prime example of a well-engineered open-source project with extensive unit tests (covering edge cases, sparse/dense variants, and GPU via array API), continuous integration across platforms, and a community-driven contribution model. In production, scikit-learn is used as a library that users import and call programmatically—not as a service. Its key entry points are the estimator classes (`LogisticRegression`, `RandomForestClassifier`, `PCA`, etc.) and utility functions (`train_test_split`, `cross_val_score`, `GridSearchCV`). Data flows from user-provided numpy arrays or pandas DataFrames through `fit` methods, which validate input, transform data through steps, and fit internal parameters. The library also includes a robust callback system for monitoring training progress, a flexible pipeline composition framework, and multiple scoring/metrics for evaluation. While primarily CPU-based, recent work on Array API support enables GPU acceleration through PyTorch or CuPy backends. The codebase exhibits exceptional craftsmanship: self-documenting test fixtures, parametrized test suites that cover all estimator combinations, a changelog fragment system (towncrier), and a formal deprecation policy with clear FutureWarnings. Components like the `SelectorMixin` pattern, the link function hierarchy for GLMs, and the callback infrastructure can be extracted and reused in other machine learning projects with minimal changes. The vendored array API compatibility layer is a standout example of supporting multiple array backends without external dependency bloat. The repository sets the benchmark for Python ML library design.
The LLM identified 4 files worth highlighting for reuse:
Provides the `CallbackSupportMixin` class and `with_callbacks` decorator that enable any estimator to support fit-time callbacks (e.g., progress bars, early stopping monitoring). The design uses a context manager (`_init_callback_context`, `subcontext`) to open/close callback listeners and propagate callbacks to nested sub-estimators. The pattern is self-contained: it defines a `FitCallback` protocol, a `RecordingCallback` for testing, and a `propagate_callback_context` function for parallel execution. This module can be extracted and applied to any Python ML framework to add callback hooks.
Implements the `SelectorMixin` abstract class that provides `transform`, `inverse_transform`, `get_feature_names_out`, and `_more_tags` out of the box. Subclasses only need to implement `_get_support_mask()`. This is a textbook example of the Template Method pattern for feature selection, enabling consistent behavior across all selectors (e.g., `RFE`, `SelectKBest`, `VarianceThreshold`). The `inverse_transform` handles sparse matrices and user-provided feature names. Easily reusable for any project needing a feature subset selection interface.
Defines a hierarchy of link functions (`BaseLink`, `IdentityLink`, `LogLink`, `LogitLink`, `MultinomialLogit`) used in Generalized Linear Models. Each link implements `link`, `inverse`, `derivative`, and `derivative_of_inverse` with proper mathematical properties. The base class uses `__init_subclass__` to enforce naming conventions and stores inverse weak references. This module is a clean, standalone abstraction for any GLM or neural network that requires differentiable link functions, and can be extracted without dependency on the rest of scikit-learn.
Implements statistical operations (mean, std, var, sum, etc.) using only the Array API standard, allowing them to work across NumPy, PyTorch, CuPy, JAX, and Dask arrays without backend-specific code. The functions detect the array namespace and dispatch to the appropriate backend implementation. This file is part of the vendored `array_api_extra` package, which provides a consistent set of operations (searching, sorting, set operations) that are missing from the base Array API standard. This pattern is reusable for any project aiming to support multiple array backends with minimal code duplication.
The divergent implementation provides a more comprehensive and organized exposure of wrapper classes by grouping them logically under a single `__init__.py` file, making it easier for users to discover and import all available wrappers. Unlike the typical implementation which may scatter imports or require explicit submodule access, this approach centralizes the API while maintaining backward compatibility. The structured import organization also improves code readability and reduces the chance of import-related errors, demonstrating a cleaner architectural approach to exposing a collection of related classes.