qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization

TL;DR

qEUBO is a new decision-theoretic acquisition function for preferential Bayesian optimization that outperforms existing methods, converges to zero simple regret under noise, and is computationally efficient.

Contribution

The paper introduces qEUBO, a novel acquisition function for PBO with theoretical optimality and proven superior empirical performance.

Findings

qEUBO is Bayes optimal in noise-free settings.

qEUBO outperforms state-of-the-art acquisition functions in experiments.

qEUBO's simple regret converges to zero at rate o(1/n).

Abstract

Preferential Bayesian optimization (PBO) is a framework for optimizing a decision maker's latent utility function using preference feedback. This work introduces the expected utility of the best option (qEUBO) as a novel acquisition function for PBO. When the decision maker's responses are noise-free, we show that qEUBO is one-step Bayes optimal and thus equivalent to the popular knowledge gradient acquisition function. We also show that qEUBO enjoys an additive constant approximation guarantee to the one-step Bayes-optimal policy when the decision maker's responses are corrupted by noise. We provide an extensive evaluation of qEUBO and demonstrate that it outperforms the state-of-the-art acquisition functions for PBO across many settings. Finally, we show that, under sufficient regularity conditions, qEUBO's Bayesian simple regret converges to zero at a rate $o(1/n)$ as the number of queries, $n$, goes to infinity. In contrast, we show that simple regret under qEI, a popular acquisition function for standard BO often used for PBO, can fail to converge to zero. Enjoying superior performance, simple computation, and a grounded decision-theoretic justification, qEUBO is a promising acquisition function for PBO.