Pareto-frontier Entropy Search with Variational Lower Bound Maximization

TL;DR

This paper introduces a novel approach for multi-objective Bayesian optimization that approximates the Pareto-frontier's truncated distribution using a mixture model optimized via variational lower bound, improving efficiency with many objectives.

Contribution

It proposes a variational lower bound maximization method to optimize the mixture distribution approximation of the Pareto-frontier's truncated distribution in MOBO.

Findings

Effective for large numbers of objectives

Improves approximation of Pareto-frontier in MOBO

Reduces computational complexity

Abstract

This study considers multi-objective Bayesian optimization (MOBO) through the information gain of the Pareto-frontier. To calculate the information gain, a predictive distribution conditioned on the Pareto-frontier plays a key role, which is defined as a distribution truncated by the Pareto-frontier. However, it is usually impossible to obtain the entire Pareto-frontier in a continuous domain, and therefore, the complete truncation cannot be known. We consider an approximation of the truncate distribution by using a mixture distribution consisting of two possible approximate truncation obtainable from a subset of the Pareto-frontier, which we call over- and under-truncation. Since the optimal balance of the mixture is unknown beforehand, we propose optimizing the balancing coefficient through the variational lower bound maximization framework, by which the approximation error of the information gain can be minimized. Our empirical evaluation demonstrates the effectiveness of the proposed method particularly when the number of objective functions is large.