Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

TL;DR

This paper introduces a goal-oriented calibration method for Gaussian process models in Bayesian optimization, improving lower-tail predictive accuracy and optimization performance.

Contribution

It proposes tcGP, a post-hoc calibration technique targeting the lower tail of GP predictive distributions, enhancing BO decision-making.

Findings

tcGP improves lower-tail calibration over standard GP models.

Enhanced calibration leads to better Bayesian optimization results.

The method maintains dense sampling in the design space.

Abstract

Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel choice and hyperparameter selection can lead to miscalibrated predictive distributions and an inappropriate exploration-exploitation trade-off. For minimization, sampling criteria such as expected improvement (EI) depend on the predictive distribution below the current best value, so lower-tail miscalibration directly affects the sampling decision. This article studies goal-oriented calibration of GP predictive distributions below a low threshold $t$ in the noiseless setting, for standard GP models with hyperparameters selected by maximum likelihood. A framework for predictive reliability below $t$ is introduced, based on two notions of spatial calibration: occurrence calibration over the design space and thresholded $\mu$-calibration on sublevel sets of the form $\{x\in\mathbb{X}, f(x)\le t\}$. Building on this framework, we propose tcGP, a post-hoc method that calibrates GP predictive distributions below~$t$, and we show that the resulting EI-based global optimization algorithm remains dense in the design space. Experiments on standard benchmarks show improved lower-tail calibration and BO performance relative to standard GP models and globally calibrated GP models.