Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs

TL;DR

This paper introduces AIRBO, a robust Bayesian Optimization method that effectively handles arbitrary input uncertainties by modeling them with Gaussian Processes enhanced by MMD, achieving state-of-the-art results.

Contribution

The paper presents a novel robust BO algorithm, AIRBO, which models arbitrary input uncertainties using MMD and accelerates inference with Nystrom approximation, providing theoretical guarantees.

Findings

Handles various input uncertainties effectively

Achieves state-of-the-art performance on synthetic and real problems

Provides theoretical regret bounds under MMD estimation error

Abstract

Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in the final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nystrom approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve state-of-the-art performance.