Batched Energy-Entropy acquisition for Bayesian Optimization

TL;DR

This paper introduces BEEBO, a novel energy-entropy based acquisition function for batch Bayesian optimization that efficiently manages exploration and exploitation, especially in heteroskedastic black-box problems.

Contribution

The paper presents BEEBO, a physics-inspired acquisition function that natively handles batch selection in Gaussian process-based Bayesian optimization, improving exploration-exploitation control.

Findings

BEEBO performs competitively with existing batch BO methods.

BEEBO effectively manages exploration and exploitation trade-offs.

The method generalizes well to heteroskedastic black-box problems.

Abstract

Bayesian optimization (BO) is an attractive machine learning framework for performing sample-efficient global optimization of black-box functions. The optimization process is guided by an acquisition function that selects points to acquire in each round of BO. In batched BO, when multiple points are acquired in parallel, commonly used acquisition functions are often high-dimensional and intractable, leading to the use of sampling-based alternatives. We propose a statistical physics inspired acquisition function for BO with Gaussian processes that can natively handle batches. Batched Energy-Entropy acquisition for BO (BEEBO) enables tight control of the explore-exploit trade-off of the optimization process and generalizes to heteroskedastic black-box problems. We demonstrate the applicability of BEEBO on a range of problems, showing competitive performance to existing methods.