Optimization with SpotOptim

TL;DR

SpotOptim is an open-source Python package for surrogate-model-based optimization of expensive black-box functions, supporting various variable types, noise handling, multi-objective optimization, parallelization, and real-time monitoring.

Contribution

It introduces a flexible, scalable optimization framework integrating surrogate modeling, multi-objective support, and advanced restart strategies, with comprehensive comparisons to existing tools.

Findings

Effective hyperparameter tuning demonstrated on neural networks.

Parallelization improves optimization efficiency on multi-core hardware.

Competitive performance compared to established optimization frameworks.

Abstract

The `spotoptim` package implements surrogate-model-based optimization of expensive black-box functions in Python. Building on two decades of Sequential Parameter Optimization (SPO) methodology, it provides a Kriging-based optimization loop with Expected Improvement, support for continuous, integer, and categorical variables, noise-aware evaluation via Optimal Computing Budget Allocation (OCBA), and multi-objective extensions. A steady-state parallelization strategy overlaps surrogate search with objective evaluation on multi-core hardware, and a success-rate-based restart mechanism detects stagnation while preserving the best solution found. The package returns scipy-compatible `OptimizeResult` objects and accepts any scikit-learn-compatible surrogate model. Built-in TensorBoard logging provides real-time monitoring of convergence and surrogate quality. This report describes the architecture and module structure of spotoptim, provides worked examples including neural network hyperparameter tuning, and compares the framework with BoTorch, Optuna, Ray Tune, BOHB, SMAC, and Hyperopt. The package is open-source.