Bayesian Optimization with Shape Constraints

TL;DR

This paper introduces a methodology to incorporate shape constraints into Bayesian optimization, enhancing its effectiveness in hyperparameter tuning and decision analysis by leveraging prior shape information.

Contribution

It presents a novel approach for integrating shape constraints into Bayesian optimization, expanding its applicability and potential accuracy in relevant domains.

Findings

Shape constraints improve optimization performance.

Methodology is effective in hyperparameter tuning.

Promising initial results for decision analysis.

Abstract

In typical applications of Bayesian optimization, minimal assumptions are made about the objective function being optimized. This is true even when researchers have prior information about the shape of the function with respect to one or more argument. We make the case that shape constraints are often appropriate in at least two important application areas of Bayesian optimization: (1) hyperparameter tuning of machine learning algorithms and (2) decision analysis with utility functions. We describe a methodology for incorporating a variety of shape constraints within the usual Bayesian optimization framework and present positive results from simple applications which suggest that Bayesian optimization with shape constraints is a promising topic for further research.