Bayesian Optimization with Expected Improvement: No Regret and the Choice of Incumbent

TL;DR

This paper provides the first theoretical analysis of the cumulative regret bounds for Bayesian optimization using expected improvement with different incumbents, demonstrating no-regret properties and guiding practitioners in noisy settings.

Contribution

It establishes the first cumulative regret upper bounds for GP-EI with BPMI and BSPMI, and analyzes the regret behavior of GP-EI with BOI for the first time.

Findings

GP-EI with BPMI and BSPMI is a no-regret algorithm for SE and Matern kernels.

GP-EI with BOI achieves sublinear regret or fast converging simple regret bounds.

Numerical experiments validate the theoretical regret bounds and guidance for choosing incumbents.

Abstract

Expected improvement (EI) is one of the most widely used acquisition functions in Bayesian optimization (BO). Despite its proven empirical success in applications, the cumulative regret upper bound of EI remains an open question. In this paper, we analyze the classic noisy Gaussian process expected improvement (GP-EI) algorithm. We consider the Bayesian setting, where the objective is a sample from a GP. Three commonly used incumbents, namely the best posterior mean incumbent (BPMI), the best sampled posterior mean incumbent (BSPMI), and the best observation incumbent (BOI) are considered as the choices of the current best value in GP-EI. We present for the first time the cumulative regret upper bounds of GP-EI with BPMI and BSPMI. Importantly, we show that in both cases, GP-EI is a no-regret algorithm for both squared exponential (SE) and Mat\'ern kernels. Further, we present for the first time that GP-EI with BOI either achieves a sublinear cumulative regret upper bound or has a fast converging noisy simple regret bound for SE and Mat\'ern kernels. Our results provide theoretical guidance to the choice of incumbent when practitioners apply GP-EI in the noisy setting. Numerical experiments are conducted to validate our findings.