A Corrected Expected Improvement Acquisition Function Under Noisy Observations

TL;DR

This paper introduces a corrected expected improvement (EI) acquisition function for Bayesian optimization that effectively accounts for noisy observations by incorporating Gaussian Process covariance information, improving performance in noisy settings.

Contribution

The authors propose a modified EI that incorporates covariance information to handle noisy observations, extending its applicability and accuracy in Bayesian optimization.

Findings

The corrected EI achieves a sublinear regret bound under heteroscedastic noise.

Empirical results show improved performance over classical EI in noisy black-box optimization tasks.

The method generalizes well to both noisy and noiseless optimization scenarios.

Abstract

Sequential maximization of expected improvement (EI) is one of the most widely used policies in Bayesian optimization because of its simplicity and ability to handle noisy observations. In particular, the improvement function often uses the best posterior mean as the best incumbent in noisy settings. However, the uncertainty associated with the incumbent solution is often neglected in many analytic EI-type methods: a closed-form acquisition function is derived in the noise-free setting, but then applied to the setting with noisy observations. To address this limitation, we propose a modification of EI that corrects its closed-form expression by incorporating the covariance information provided by the Gaussian Process (GP) model. This acquisition function specializes to the classical noise-free result, and we argue should replace that formula in Bayesian optimization software packages, tutorials, and textbooks. This enhanced acquisition provides good generality for noisy and noiseless settings. We show that our method achieves a sublinear convergence rate on the cumulative regret bound under heteroscedastic observation noise. Our empirical results demonstrate that our proposed acquisition function can outperform EI in the presence of noisy observations on benchmark functions for black-box optimization, as well as on parameter search for neural network model compression.