The reparameterization trick for acquisition functions

TL;DR

This paper introduces a reparameterization approach for acquisition functions in Bayesian optimization, enabling gradient-based optimization and more efficient parallel selection strategies.

Contribution

It reformulates many acquisition functions as Gaussian integrals using the reparameterization trick, facilitating their optimization.

Findings

Enables gradient-based optimization of acquisition functions.

Derives an efficient Monte Carlo estimator for parallel upper confidence bound.

Improves optimization efficiency in parallel Bayesian optimization.

Abstract

Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guide the search process. Maximizing acquisition functions yields the best performance; unfortunately, this ideal is difficult to achieve since optimizing acquisition functions per se is frequently non-trivial. This statement is especially true in the parallel setting, where acquisition functions are routinely non-convex, high-dimensional, and intractable. Here, we demonstrate how many popular acquisition functions can be formulated as Gaussian integrals amenable to the reparameterization trick and, ensuingly, gradient-based optimization. Further, we use this reparameterized representation to derive an efficient Monte Carlo estimator for the upper confidence bound acquisition function in the context of parallel selection.