Constrained Bayesian optimization with merit functions

TL;DR

This paper introduces constrained Bayesian optimization algorithms that incorporate merit functions to improve efficiency, handle infeasible samples, and extend existing methods like ECI, demonstrated through synthetic and engineering applications.

Contribution

The paper proposes new CBO algorithms using merit functions, providing closed-form acquisition functions and a unified approach extending ECI, enhancing efficiency and practicality.

Findings

Merit function-based CBO algorithms outperform traditional methods.

The proposed algorithms handle infeasible initial samples effectively.

Numerical experiments validate improved efficiency and applicability.

Abstract

Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions, such as the penalty merit function, in acquisition functions, inspired by nonlinear optimization methods, e.g., sequential quadratic programming. Merit functions measure the potential progress of both the objective and constraint functions, thus increasing algorithmic efficiency and allowing infeasible initial samples. The acquisition functions with merit functions are relaxed to have closed forms, making its implementation readily available wherever Bayesian optimization is. We further propose a unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach. We demonstrate the effectiveness and efficiency of the proposed algorithms through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.