Consecutive Preferential Bayesian Optimization

TL;DR

This paper introduces Consecutive Preferential Bayesian Optimization, a method that reduces costs by constraining comparisons to previous candidates and models perceptual indifference, improving optimization accuracy in costly or ambiguous feedback scenarios.

Contribution

It extends preference-based optimization to explicitly incorporate production costs and perceptual indifference, enhancing efficiency and accuracy.

Findings

Significant cost reduction in high-cost evaluation setups.

Improved optimization accuracy with perceptual indifference modeling.

Effective selection of informative configurations using an adapted acquisition strategy.

Abstract

Preferential Bayesian optimization allows optimization of objectives that are either expensive or difficult to measure directly, by relying on a minimal number of comparative evaluations done by a human expert. Generating candidate solutions for evaluation is also often expensive, but this cost is ignored by existing methods. We generalize preference-based optimization to explicitly account for production and evaluation costs with Consecutive Preferential Bayesian Optimization, reducing production cost by constraining comparisons to involve previously generated candidates. We also account for the perceptual ambiguity of the oracle providing the feedback by incorporating a Just-Noticeable Difference threshold into a probabilistic preference model to capture indifference to small utility differences. We adapt an information-theoretic acquisition strategy to this setting, selecting new configurations that are most informative about the unknown optimum under a preference model accounting for the perceptual ambiguity. We empirically demonstrate a notable increase in accuracy in setups with high production costs or with indifference feedback.