Mono-surrogate vs Multi-surrogate in Multi-objective Bayesian Optimisation

TL;DR

This paper compares mono-surrogate and multi-surrogate Bayesian optimization approaches for multi-objective problems, proposing a multi-surrogate method that overcomes limitations of the mono-surrogate approach by modeling each objective separately and using a Generalised extreme value distribution.

Contribution

The paper introduces a multi-surrogate Bayesian optimization method that models each objective individually and uses a Generalised extreme value distribution to better approximate the scalarising function's distribution.

Findings

Multi-surrogate approach outperforms mono-surrogate on benchmarks.

Scalarising function distribution is not Gaussian, as previously assumed.

Multi-surrogate method effectively handles diverse objective landscapes.

Abstract

Bayesian optimisation (BO) has been widely used to solve problems with expensive function evaluations. In multi-objective optimisation problems, BO aims to find a set of approximated Pareto optimal solutions. There are typically two ways to build surrogates in multi-objective BO: One surrogate by aggregating objective functions (by using a scalarising function, also called mono-surrogate approach) and multiple surrogates (for each objective function, also called multi-surrogate approach). In both approaches, an acquisition function (AF) is used to guide the search process. Mono-surrogate has the advantage that only one model is used, however, the approach has two major limitations. Firstly, the fitness landscape of the scalarising function and the objective functions may not be similar. Secondly, the approach assumes that the scalarising function distribution is Gaussian, and thus a closed-form expression of the AF can be used. In this work, we overcome these limitations by building a surrogate model for each objective function and show that the scalarising function distribution is not Gaussian. We approximate the distribution using Generalised extreme value distribution. The results and comparison with existing approaches on standard benchmark and real-world optimisation problems show the potential of the multi-surrogate approach.