MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation

TL;DR

MBORE introduces a novel multi-objective Bayesian optimisation method that leverages density-ratio estimation and classifiers, outperforming traditional Gaussian process-based BO especially in high-dimensional and real-world problems.

Contribution

This work extends BO with density-ratio estimation to the multi-objective setting, enabling scalable and effective optimisation without Gaussian processes.

Findings

MBORE performs as well as or better than BO on synthetic benchmarks.

MBORE outperforms BO on high-dimensional problems.

MBORE is effective on real-world optimisation tasks.

Abstract

Optimisation problems often have multiple conflicting objectives that can be computationally and/or financially expensive. Mono-surrogate Bayesian optimisation (BO) is a popular model-based approach for optimising such black-box functions. It combines objective values via scalarisation and builds a Gaussian process (GP) surrogate of the scalarised values. The location which maximises a cheap-to-query acquisition function is chosen as the next location to expensively evaluate. While BO is an effective strategy, the use of GPs is limiting. Their performance decreases as the problem input dimensionality increases, and their computational complexity scales cubically with the amount of data. To address these limitations, we extend previous work on BO by density-ratio estimation (BORE) to the multi-objective setting. BORE links the computation of the probability of improvement acquisition function to that of probabilistic classification. This enables the use of state-of-the-art classifiers in a BO-like framework. In this work we present MBORE: multi-objective Bayesian optimisation by density-ratio estimation, and compare it to BO across a range of synthetic and real-world benchmarks. We find that MBORE performs as well as or better than BO on a wide variety of problems, and that it outperforms BO on high-dimensional and real-world problems.