Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes

TL;DR

This paper advances preferential Bayesian optimization by improving the modeling of skew Gaussian process posteriors, enhancing computational efficiency, and providing more accurate preference predictions through novel MCMC techniques and practical algorithms.

Contribution

It introduces a new method that combines high efficiency and low sample complexity for preferential BO using skew Gaussian processes, addressing previous computational and accuracy limitations.

Findings

Gaussian approximation can be inaccurate for duel prediction

Enhanced MCMC methods improve skew GP estimation

Proposed method achieves better efficiency and accuracy in experiments

Abstract

We study preferential Bayesian optimization (BO) where reliable feedback is limited to pairwise comparison called duels. An important challenge in preferential BO, which uses the preferential Gaussian process (GP) model to represent flexible preference structure, is that the posterior distribution is a computationally intractable skew GP. The most widely used approach for preferential BO is Gaussian approximation, which ignores the skewness of the true posterior. Alternatively, Markov chain Monte Carlo (MCMC) based preferential BO is also proposed. In this work, we first verify the accuracy of Gaussian approximation, from which we reveal the critical problem that the predictive probability of duels can be inaccurate. This observation motivates us to improve the MCMC-based estimation for skew GP, for which we show the practical efficiency of Gibbs sampling and derive the low variance MC estimator. However, the computational time of MCMC can still be a bottleneck in practice. Towards building a more practical preferential BO, we develop a new method that achieves both high computational efficiency and low sample complexity, and then demonstrate its effectiveness through extensive numerical experiments.