Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization

TL;DR

This paper introduces a heteroscedastic noise model for Preferential Bayesian Optimization that accounts for input-dependent user uncertainty, improving performance in human-in-the-loop scenarios.

Contribution

It proposes a novel anchor-based heteroscedastic noise model, integrating user-provided reliable examples into PBO and deriving risk-averse acquisition functions.

Findings

Improved risk-adjusted performance on synthetic and human-preference datasets.

Risk-adjusted EUBO maintains one-step Bayes-optimality up to a constant.

Anchor placement significantly influences method effectiveness.

Abstract

Preferential Bayesian optimization (PBO) learns latent utilities from pairwise comparisons, but most existing methods assume homoscedastic comparison noise. This is inadequate in human-in-the-loop settings, where a user may compare some designs reliably and others only hesitantly. We propose a heteroscedastic noise model for PBO: before optimization, the user provides a small set of reliable examples, called anchors, and a kernel density estimator (KDE) turns these anchors into an input-dependent map of user uncertainty. We incorporate this map into preferential GP surrogates and derive risk-averse acquisition functions that trade off utility and ease of comparison. We further show that a risk-adjusted variant of the popular expected utility of the best option (EUBO) preserves the one-step Bayes-optimality guarantee up to an additive constant, and that under an idealized i.i.d. anchor model the KDE estimator enjoys standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets show improved risk-adjusted performance and clarify how anchor placement affects the method.