Elicitation-Augmented Bayesian Optimization

TL;DR

This paper introduces a Bayesian optimization method that incorporates human expertise through pairwise comparisons, improving sample efficiency when expert input is cost-effective and maintaining performance when it is noisy or costly.

Contribution

It develops a principled, cost-aware framework for combining direct observations and pairwise queries in human-in-the-loop Bayesian optimization.

Findings

Significantly improves sample-efficiency with cheap pairwise queries.

Recovers standard BO performance when pairwise queries are costly or noisy.

Provides a theoretical foundation for elicitation-augmented Bayesian optimization.

Abstract

Human-in-the-loop Bayesian optimization (HITL BO) methods utilize human expertise to improve the sample-efficiency of BO. Most HITL BO methods assume that a domain expert can quantify their knowledge, for instance by pinpointing query locations or specifying their prior beliefs about the location of the maximum as a probability distribution. However, since human expertise is often tacit and cannot be explicitly quantified, we consider a setting where domain knowledge of an expert is elicited via pairwise comparisons of designs. We interpret the expert's pairwise judgements as noisy evidence about the values of the observable objective function and develop a principled method for combining the information obtained via direct observations and pairwise queries. Specifically, we derive a cost-aware value-of-information acquisition function that balances direct observations against pairwise queries. The proposed method approaches the convex hull of the trajectories of the individual information sources: when pairwise queries are cheap it substantially improves sample-efficiency over observation-only BO, and when pairwise queries are costly or noisy, it recovers the performance of standard BO by relying on direct observations alone.