Mixture-Model Preference Learning for Many-Objective Bayesian Optimization

TL;DR

This paper introduces a Bayesian framework for preference learning in many-objective optimization, modeling multiple latent archetypes to better capture heterogeneous human preferences and improve optimization performance.

Contribution

It proposes a mixture-model approach with hybrid queries and provides theoretical guarantees, outperforming standard methods on benchmarks.

Findings

Outperforms baseline methods on synthetic and real-world benchmarks.

Reveals structure in preferences that regret alone cannot capture.

Provides a simple regret guarantee for the mixture-aware Bayesian optimization.

Abstract

Preference-based many-objective optimization faces two obstacles: an expanding space of trade-offs and heterogeneous, context-dependent human value structures. Towards this, we propose a Bayesian framework that learns a small set of latent preference archetypes rather than assuming a single fixed utility function, modelling them as components of a Dirichlet-process mixture with uncertainty over both archetypes and their weights. To query efficiently, we designing hybrid queries that target information about (i) mode identity and (ii) within-mode trade-offs. Under mild assumptions, we provide a simple regret guarantee for the resulting mixture-aware Bayesian optimization procedure. Empirically, our method outperforms standard baselines on synthetic and real-world many-objective benchmarks, and mixture-aware diagnostics reveal structure that regret alone fails to capture.