SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

TL;DR

SMAC3 is a flexible Bayesian Optimization toolkit designed to efficiently tune hyperparameters and solve various optimization problems, supporting diverse use cases with pre-configured options and a user-friendly interface.

Contribution

It introduces a versatile, open-source Bayesian Optimization package that simplifies hyperparameter tuning and global optimization tasks across multiple scenarios.

Findings

Improves hyperparameter tuning efficiency with fewer evaluations

Supports multiple optimization use cases with pre-set configurations

Available as an open-source package under BSD license

Abstract

Algorithm parameters, in particular hyperparameters of machine learning algorithms, can substantially impact their performance. To support users in determining well-performing hyperparameter configurations for their algorithms, datasets and applications at hand, SMAC3 offers a robust and flexible framework for Bayesian Optimization, which can improve performance within a few evaluations. It offers several facades and pre-sets for typical use cases, such as optimizing hyperparameters, solving low dimensional continuous (artificial) global optimization problems and configuring algorithms to perform well across multiple problem instances. The SMAC3 package is available under a permissive BSD-license at https://github.com/automl/SMAC3.