Optimal Initialization of Batch Bayesian Optimization

TL;DR

This paper introduces a novel batch-design acquisition function called Minimal Terminal Variance (MTV) for Bayesian optimization, which optimizes batch selection by minimizing the variance of quality estimates, improving efficiency in experimental settings.

Contribution

The paper proposes MTV, a new acquisition function that optimizes batch design using a variance minimization criterion, applicable to both initial and subsequent batches in Bayesian optimization.

Findings

MTV outperforms existing BBO methods in numerical experiments.

MTV effectively designs informative batches by minimizing variance.

Applicable to both initialization and ongoing batch selection.

Abstract

Field experiments and computer simulations are effective but time-consuming methods of measuring the quality of engineered systems at different settings. To reduce the total time required, experimenters may employ Bayesian optimization, which is parsimonious with measurements, and take measurements of multiple settings simultaneously, in a batch. In practice, experimenters use very few batches, thus, it is imperative that each batch be as informative as possible. Typically, the initial batch in a Batch Bayesian Optimization (BBO) is constructed from a quasi-random sample of settings values. We propose a batch-design acquisition function, Minimal Terminal Variance (MTV), that designs a batch by optimization rather than random sampling. MTV adapts a design criterion function from Design of Experiments, called I-Optimality, which minimizes the variance of the post-evaluation estimates of quality, integrated over the entire space of settings. MTV weights the integral by the probability that a setting is optimal, making it able to design not only an initial batch but all subsequent batches, as well. Applicability to both initialization and subsequent batches is novel among acquisition functions. Numerical experiments on test functions and simulators show that MTV compares favorably to other BBO methods.