A Similarity Measure of Gaussian Process Predictive Distributions

TL;DR

This paper introduces a similarity measure for Gaussian process predictive distributions to identify when multiple GPs model correlated functions, enabling more efficient Bayesian optimization by reducing redundant models.

Contribution

The paper proposes a novel similarity metric for GP predictive distributions, facilitating the detection of correlated functions and improving efficiency in multi-objective Bayesian optimization.

Findings

GP predictive distributions can be effectively compared using the proposed similarity measure.

One GP can model multiple correlated functions, reducing computational complexity.

Empirical results on synthetic and benchmark data validate the usefulness of the similarity metric.

Abstract

Some scenarios require the computation of a predictive distribution of a new value evaluated on an objective function conditioned on previous observations. We are interested on using a model that makes valid assumptions on the objective function whose values we are trying to predict. Some of these assumptions may be smoothness or stationarity. Gaussian process (GPs) are probabilistic models that can be interpreted as flexible distributions over functions. They encode the assumptions through covariance functions, making hypotheses about new data through a predictive distribution by being fitted to old observations. We can face the case where several GPs are used to model different objective functions. GPs are non-parametric models whose complexity is cubic on the number of observations. A measure that represents how similar is one GP predictive distribution with respect to another would be useful to stop using one GP when they are modelling functions of the same input space. We are really inferring that two objective functions are correlated, so one GP is enough to model both of them by performing a transformation of the prediction of the other function in case of inverse correlation. We show empirical evidence in a set of synthetic and benchmark experiments that GPs predictive distributions can be compared and that one of them is enough to predict two correlated functions in the same input space. This similarity metric could be extremely useful used to discard objectives in Bayesian many-objective optimization.