Generative Bayesian Optimization: Generative Models as Acquisition Functions

TL;DR

This paper introduces a novel approach that uses generative models as acquisition functions in batch Bayesian optimization, enabling efficient optimization in high-dimensional, non-continuous, and combinatorial spaces.

Contribution

It proposes a general method to train generative models with utility signals directly, bypassing surrogate models, and demonstrates theoretical convergence and practical effectiveness.

Findings

Generative models can approximate optimal acquisition distributions asymptotically.

The method scales to large batches and high-dimensional, non-continuous spaces.

Experimental results show improved performance on complex optimization tasks.

Abstract

We present a general strategy for turning generative models into candidate solution samplers for batch Bayesian optimization (BO). The use of generative models for BO enables large batch scaling as generative sampling, optimization of non-continuous design spaces, and high-dimensional and combinatorial design. Inspired by the success of direct preference optimization (DPO), we show that one can train a generative model with noisy, simple utility values directly computed from observations to then form proposal distributions whose densities are proportional to the expected utility, i.e., BO's acquisition function values. Furthermore, this approach is generalizable beyond preference-based feedback to general types of reward signals and loss functions. This perspective avoids the construction of surrogate (regression or classification) models, common in previous methods that have used generative models for black-box optimization. Theoretically, we show that the generative models within the BO process follow a sequence of distributions which asymptotically approximate an optimal target under certain conditions. We also evaluate the performance through experiments on challenging optimization problems involving large batches in high dimensions.