Ray is a distributed compute framework that unifies task/actor execution, data processing, model training, hyperparameter tuning, RL, and model serving under a single runtime. The core is written in C++ (raylet, GCS, object store, scheduling) with a Python frontend, communicating via gRPC and Arrow-compatible plasma object storage. Production entry points include `ray.init()` for cluster bootstrap, `@ray.remote` for task/actor offload, Ray Serve for model deployment (FastAPI/DAG/gang-scheduling patterns), Ray Data for streaming ETL, Ray Train/Tune for distributed training and hyperparameter search, and RLlib for reinforcement learning. The architecture is a hybrid: a distributed systems platform (scheduler, object store, autoscaler) layered with high-level AI libraries, all built via Bazel with extensive CI (Docker images, build matrices, release tests). It is not a data research repo, though Ray Data and RLlib contain research-grade data pipeline and RL primitives; the dominant character is a production-grade application framework.
auto-sklearn is an automated machine learning library offering a scikit-learn-compatible interface for classification and regression. It performs hyperparameter optimization via Bayesian optimization (SMAC), integrates meta-learning to warm-start the search from prior runs, and builds ensembles using Caruana's ensemble selection. The codebase is modular: pipeline components (classifiers, regressors, preprocessors) are defined with ConfigSpace and can be extended via a third-party component system. Parallel execution uses dask, and custom metrics, cross-validation splitters, and early stopping are supported. The repository also contains a substantial research component—precomputed meta-features and algorithm run data stored in ARFF format—and scripts for reproducing experiments. The architecture is well-tested with a comprehensive test suite, though some dependencies are pinned to older versions and there is moderate technical drift.