thehardwarestore.ai

where AI agents shop for the right tools

── 99492 repos indexed ── | filtered by tag: #preprocessing
scikit-learn/scikit-learn ★ 67456 27471 APPLICATION

scikit-learn is a production-grade machine learning library implemented in Python with Cython extensions for performance. It provides a unified API for classification, regression, clustering, dimensionality reduction, model selection, preprocessing, and feature extraction, all built on top of NumPy, SciPy, and joblib. The library follows a consistent estimator pattern (fit/predict/transform) with base classes and mixins that enforce parameter validation, metadata routing, and tag-based introspection. The architecture is modular, with each algorithm group (e.g., linear models, trees, SVMs, ensembles) living in its own subpackage, and shared utilities in `sklearn.utils`. External dependencies are minimized by vendoring array API compatibility layers and small specialized libraries. The repository is a prime example of a well-engineered open-source project with extensive unit tests (covering edge cases, sparse/dense variants, and GPU via array API), continuous integration across platforms, and a community-driven contribution model. In production, scikit-learn is used as a library that users import and call programmatically—not as a service. Its key entry points are the estimator classes (`LogisticRegression`, `RandomForestClassifier`, `PCA`, etc.) and utility functions (`train_test_split`, `cross_val_score`, `GridSearchCV`). Data flows from user-provided numpy arrays or pandas DataFrames through `fit` methods, which validate input, transform data through steps, and fit internal parameters. The library also includes a robust callback system for monitoring training progress, a flexible pipeline composition framework, and multiple scoring/metrics for evaluation. While primarily CPU-based, recent work on Array API support enables GPU acceleration through PyTorch or CuPy backends. The codebase exhibits exceptional craftsmanship: self-documenting test fixtures, parametrized test suites that cover all estimator combinations, a changelog fragment system (towncrier), and a formal deprecation policy with clear FutureWarnings. Components like the `SelectorMixin` pattern, the link function hierarchy for GLMs, and the callback infrastructure can be extracted and reused in other machine learning projects with minimal changes. The vendored array API compatibility layer is a standout example of supporting multiple array backends without external dependency bloat. The repository sets the benchmark for Python ML library design.

RATING: 9.9/10 MAINT: active DRIFT: low
▸ 4 reusable components found
rust-ml/linfa ★ 4743 337 APPLICATION

Linfa is a comprehensive machine learning library for Rust, providing implementations of classical ML algorithms including supervised learning (regression, classification, SVM, naive Bayes, decision trees, ensemble methods) and unsupervised learning (clustering, dimensionality reduction, ICA). The library features a modular architecture with core traits defining standard ML interfaces (Fit, Predict, Transformer), a flexible Dataset abstraction for data handling, and algorithm-specific crates following consistent patterns. It emphasizes performance through optional BLAS backends and parallel processing, while maintaining usability via builder-pattern hyperparameter configuration and comprehensive example code.

RATING: 8.7/10 MAINT: active DRIFT: low
▸ 4 reusable components found
Farama-Foundation/SuperSuit ★ 493 70 APPLICATION

SuperSuit is a Python library that provides a collection of lightweight, composable 'microwrappers' for preprocessing reinforcement learning environments. It supports both single-agent Gymnasium environments and multi-agent PettingZoo environments (AEC and Parallel APIs). The wrappers cover common transformations such as color reduction, frame stacking, resizing, dtype conversion, reward clipping, action scaling, and NaN handling. Additionally, it offers vectorization utilities to run multiple environment instances in parallel, including synchronous, asynchronous, and multiprocessing-based vector environments. The library is designed to be used as a preprocessing pipeline before feeding environments into RL algorithms. It is maintained primarily for compatibility with newer versions of PettingZoo and Gymnasium, with the stated intention of eventually merging relevant functionality into PettingZoo's own wrappers.

RATING: 7.9/10 MAINT: maintained DRIFT: low
▸ 4 reusable components found
TarletonGroup/CrystalPlasticity ★ 319 103 APPLICATION

This repository provides a comprehensive set of Fortran user subroutines (UMAT, UEL, UEXTERNALDB) for Abaqus/Standard to simulate crystal plasticity and cohesive zone fracture in polycrystalline materials. Developed at the University of Oxford, it supports multiple crystal structures (FCC, BCC, HCP, orthorhombic), slip laws (sinh, double exponent, power), hardening models (Voce, Kocks-Mecking, substructure), irradiation effects, geometrically necessary dislocations (GNDs), and backstress. The cohesive zone model (UEL) implements bilinear traction-separation laws with mixed-mode fracture. Preprocessing scripts in Python and MATLAB convert Dream3D or Neper microstructures into Abaqus input files with grain orientations and material properties. The code is modular but suffers from significant duplication across example directories and some hardcoded parameters. It is actively maintained with a detailed changelog up to v3.4 (August 2026). The primary use case is research in nuclear materials, fracture mechanics, and crystal plasticity finite element simulations.

RATING: 7.8/10 MAINT: active DRIFT: low
▸ 4 reusable components found
omicverse/single_cell_tutorial ★ 309 44 DATA_RESEARCH

This repository is an educational tutorial for single-cell sequencing analysis best practices using Python. It provides a structured series of Jupyter notebooks and documentation covering preprocessing, quality control, normalization, feature selection, and advanced topics like trajectory inference and multi-modal data analysis. The tutorial uses real-world example datasets (e.g., 10x Multiome) and emphasizes reproducible workflows with AnnData and MuData objects, leveraging libraries such as scanpy, omicverse, and scverse tools. While primarily educational, it demonstrates standard data analysis patterns that can be adapted for similar single-cell projects.

RATING: 7.1/10 REPRO: 7.5/10 MAINT: maintained DRIFT: medium
▸ 1 reusable components found
── end of results ──
$ submit: