Efficient Contextual Preferential Bayesian Optimization with Historical Examples

TL;DR

This paper introduces an offline Bayesian optimization method that leverages historical examples and expert knowledge to efficiently learn utility functions, reducing sample needs and performing well with biased data.

Contribution

It presents a novel utility learning approach that integrates historical data and expert insights within a Bayesian framework for more efficient multi-objective optimization.

Findings

Outperforms Gaussian processes and BOPE in four domains

Effective with biased samples and limited expert input

Reduces sample requirements for utility learning

Abstract

State-of-the-art multi-objective optimization often assumes a known utility function, learns it interactively, or computes the full Pareto front-each requiring costly expert input.~Real-world problems, however, involve implicit preferences that are hard to formalize. To reduce expert involvement, we propose an offline, interpretable utility learning method that uses expert knowledge, historical examples, and coarse information about the utility space to reduce sample requirements. We model uncertainty via a full Bayesian posterior and propagate it throughout the optimization process. Our method outperforms standard Gaussian processes and BOPE across four domains, showing strong performance even with biased samples, as encountered in the real-world, and limited expert input.