A Framework for Nonlinearly-Constrained Gradient-Enhanced Local Bayesian Optimization with Comparisons to Quasi-Newton Optimizers

TL;DR

This paper introduces two novel methods for nonlinearly-constrained local Bayesian optimization, enabling deeper convergence and fewer function evaluations compared to traditional quasi-Newton methods.

Contribution

The paper develops two new constrained Bayesian optimization techniques that handle nonlinear equality constraints and improve convergence efficiency.

Findings

Both methods enable deeper convergence on constrained problems.

Fewer function evaluations needed compared to quasi-Newton optimizers.

Methods perform similarly, with the second method being more user-friendly to tune.

Abstract

Bayesian optimization is a popular and versatile approach that is well suited to solve challenging optimization problems. Their popularity comes from their effective minimization of expensive function evaluations, their capability to leverage gradients, and their efficient use of noisy data. Bayesian optimizers have commonly been applied to global unconstrained problems, with limited development for many other classes of problems. In this paper, two alternative methods are developed that enable rapid and deep convergence of nonlinearly-constrained local optimization problems using a Bayesian optimizer. The first method uses an exact augmented Lagrangian and the second augments the minimization of the acquisition function to contain additional constraints. Both of these methods can be applied to nonlinear equality constraints, unlike most previous methods developed for constrained Bayesian optimizers. The new methods are applied with a gradient-enhanced Bayesian optimizer and enable deeper convergence for three nonlinearly-constrained unimodal optimization problems than previously developed methods for constrained Bayesian optimization. In addition, both new methods enable the Bayesian optimizer to reach a desired tolerance with fewer function evaluations than popular quasi-Newton optimizers from SciPy and MATLAB for unimodal problems with 2 to 30 variables. The Bayesian optimizer had similar results using both methods. It is recommended that users first try using the second method, which adds constraints to the acquisition function minimization, since its parameters are more intuitive to tune for new problems.