Ensemble Distributionally Robust Bayesian Optimisation

TL;DR

This paper introduces a new ensemble-based Bayesian optimisation algorithm that handles distributional uncertainty in contexts, providing theoretical guarantees and demonstrating empirical effectiveness.

Contribution

It proposes a novel, computationally efficient ensemble distributionally robust Bayesian optimisation method with improved regret bounds.

Findings

Achieves sublinear regret bounds better than existing methods.

Empirical results align with theoretical guarantees.

Effectively manages continuous context in optimisation.

Abstract

We study zeroth-order optimisation under context distributional uncertainty, a setting commonly tackled using Bayesian optimisation (BO). A prevailing strategy to make BO more robust to the complex and noisy nature of data is to employ an ensemble as the surrogate model, thereby mitigating the weaknesses of any single model. In this study, we propose a novel algorithm for Ensemble Distributionally Robust Bayesian Optimisation that remains computationally tractable while managing continuous context. We obtain theoretical sublinear regret bounds, improving current state-of-the-art results. We show that our method's empirical behaviour aligns with its theoretical guarantees.