Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation

TL;DR

This paper introduces BALLET, a Bayesian optimization framework that adaptively identifies and focuses on high-confidence regions of interest, improving efficiency and reducing hyperparameter tuning in high-dimensional, non-stationary problems.

Contribution

BALLET is a novel adaptive filtering framework that uses dual Gaussian processes to identify and optimize within relevant regions, enhancing BO performance in complex scenarios.

Findings

BALLET effectively shrinks the search space in high-dimensional problems.

It achieves tighter regret bounds compared to standard BO methods.

Empirical results show improved optimization efficiency on real-world tasks.

Abstract

We study Bayesian optimization (BO) in high-dimensional and non-stationary scenarios. Existing algorithms for such scenarios typically require extensive hyperparameter tuning, which limits their practical effectiveness. We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest (ROI) as a superlevel-set of a nonparametric probabilistic model such as a Gaussian process (GP). Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods. The key idea is to use two probabilistic models: a coarse GP to identify the ROI, and a localized GP for optimization within the ROI. We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO without ROI filtering. We demonstrate empirically the effectiveness of BALLET on both synthetic and real-world optimization tasks.