Robust Bayesian Optimisation with Unbounded Corruptions

TL;DR

This paper introduces a robust Bayesian Optimization method that withstands unbounded corruptions by modeling an adversary with a frequency-bound budget, achieving sublinear regret even with infinite-magnitude outliers.

Contribution

The paper proposes RCGP-UCB, a novel algorithm combining UCB with a Robust Conjugate Gaussian Process, capable of handling unbounded corruptions in Bayesian Optimization.

Findings

Achieves sublinear regret with up to O(T^{1/4}) corruptions.

Handles corruptions with potentially infinite magnitude.

Maintains standard regret bounds in the absence of outliers.

Abstract

Bayesian Optimization is critically vulnerable to extreme outliers. Existing provably robust methods typically assume a bounded cumulative corruption budget, which makes them defenseless against even a single corruption of sufficient magnitude. To address this, we introduce a new adversary whose budget is only bounded in the frequency of corruptions, not in their magnitude. We then derive RCGP-UCB, an algorithm coupling the famous upper confidence bound (UCB) approach with a Robust Conjugate Gaussian Process (RCGP). We present stable and adaptive versions of RCGP-UCB, and prove that they achieve sublinear regret in the presence of up to $O(T^{1/4})$ and $O(T^{1/7})$ corruptions with possibly infinite magnitude. This robustness comes at near zero cost: without outliers, RCGP-UCB's regret bounds match those of the standard GP-UCB algorithm.