Choosing a Suitable Acquisition Function for Batch Bayesian Optimization: Comparison of Serial and Monte Carlo Approaches

TL;DR

This study compares serial and Monte Carlo batch acquisition functions in Bayesian optimization, finding qUCB most effective for noisy, low-dimensional black-box functions in experimental settings.

Contribution

The paper provides a comparative analysis of batch acquisition functions, recommending qUCB for efficient optimization in practical, low-dimensional experimental scenarios.

Findings

qUCB outperforms qlogEI in noiseless conditions.

qUCB and UCB/LP perform well without noise, with qUCB being most robust.

Empirical results confirm qUCB's effectiveness in real experimental data.

Abstract

Batch Bayesian optimization is widely used for optimizing expensive experimental processes when several samples can be tested together to save time or cost. A central decision in designing a Bayesian optimization campaign to guide experiments is the choice of a batch acquisition function when little or nothing is known about the landscape of the "black box" function to be optimized. To inform this decision, we first compare the performance of serial and Monte Carlo batch acquisition functions on two mathematical functions that serve as proxies for typical materials synthesis and processing experiments. The two functions, both in six dimensions, are the Ackley function, which epitomizes a "needle-in-haystack" search, and the Hartmann function, which exemplifies a "false optimum" problem. Our study evaluates the serial upper confidence bound with local penalization (UCB/LP) batch acquisition policy against Monte Carlo-based parallel approaches: q-log expected improvement (qlogEI) and q-upper confidence bound (qUCB), where q is the batch size. Tests on Ackley and Hartmann show that UCB/LP and qUCB perform well in noiseless conditions, both outperforming qlogEI. For the Hartmann function with noise, all Monte Carlo functions achieve faster convergence with less sensitivity to initial conditions compared to UCB/LP. We then confirm the findings on an empirical regression model built from experimental data in maximizing power conversion efficiency of flexible perovskite solar cells. Our results suggest that when empirically optimizing a "black-box" function in less than or equal to six dimensions with no prior knowledge of the landscape or noise characteristics, qUCB is best suited as the default to maximize confidence in the modeled optimum while minimizing the number of expensive samples needed.