$ show bayesian-optimization/BayesianOptimization
This is a mature Python library for global optimization of expensive black-box functions using Bayesian optimization. It provides the `BayesianOptimization` class that iteratively fits a Gaussian process regressor (via scikit-learn) to observed function evaluations and uses an acquisition function (Upper Confidence Bound, Expected Improvement, Probability of Improvement, or combined) to propose the next point to evaluate. The library supports constrained optimization, domain reduction, mixed continuous/integer/categorical parameters, asynchronous/parallel optimization, and state persistence. Its architecture is modular: acquisition functions are organized in a strategy pattern, parameters are polymorphic with a base class for custom types, constraints are modeled with separate GPs, and domain reduction is implemented as a transformer. The core algorithm is well-tested and documented, with examples ranging from toy functions to hyperparameter tuning. It is not a research notebook collection or data pipeline; it is a production-grade library intended for integration into optimization workflows.
The LLM identified 2 files worth highlighting for reuse:
Defines a class hierarchy for optimization parameters: `BayesParameter` base class with subclasses `FloatParameter`, `IntParameter`, `CategoricalParameter`, and a `CustomParameter` placeholder. Each parameter implements `to_float` (map to continuous domain), `to_param` (inverse), `kernel_transform` (modify input before GP kernel evaluation to handle discreteness), `random_sample`, and `to_string`. The `wrap_kernel` function takes a kernel and parameter list and returns a wrapped kernel that applies `kernel_transform` during evaluation, enabling the GP to handle integer and categorical variables. This module is a clean, self-contained abstraction for parameter handling that could be reused in other optimization frameworks with minimal adaptation.
Implements acquisition functions for Bayesian optimization using a strategy pattern. `AcquisitionFunction` is an abstract base class requiring a `suggest` method. Concrete implementations: `UpperConfidenceBound`, `ProbabilityOfImprovement`, `ExpectedImprovement`. Meta-acquisition functions `ConstantLiar` and `GPHedge` wrap others to support parallel/portfolio optimization. The `suggest` method uses scipy's L-BFGS-B for continuous domains and differential evolution for mixed domains, handling thousands of random starts. The code is well-structured, decoupled from the optimizer (only depends on `TargetSpace` and utility functions), and demonstrates careful numerical handling.