Gaussian Process regression over discrete probability measures: on the non-stationarity relation between Euclidean and Wasserstein Squared Exponential Kernels

TL;DR

This paper investigates Gaussian Process regression over probability measures using Wasserstein distance, revealing non-stationarity issues and proposing an algebraic transformation to effectively model non-Euclidean inputs.

Contribution

It uncovers the non-stationarity relationship between Wasserstein and Euclidean kernels and introduces a simple algebraic transformation for Gaussian Process modeling on probability measures.

Findings

Identifies non-stationarity as a key issue in Wasserstein-based Gaussian Processes.

Proposes an algebraic transformation to address non-stationarity.

Demonstrates improved modeling of probability measures with the proposed method.

Abstract

Gaussian Process regression is a kernel method successfully adopted in many real-life applications. Recently, there is a growing interest on extending this method to non-Euclidean input spaces, like the one considered in this paper, consisting of probability measures. Although a Positive Definite kernel can be defined by using a suitable distance -- the Wasserstein distance -- the common procedure for learning the Gaussian Process model can fail due to numerical issues, arising earlier and more frequently than in the case of an Euclidean input space and, as demonstrated in this paper, that cannot be avoided by adding artificial noise (nugget effect) as usually done. This paper uncovers the main reason of these issues, that is a non-stationarity relationship between the Wasserstein-based squared exponential kernel and its Euclidean-based counterpart. As a relevant result, the Gaussian Process model is learned by assuming the input space as Euclidean and then an algebraic transformation, based on the uncovered relation, is used to transform it into a non-stationary and Wasserstein-based Gaussian Process model over probability measures. This algebraic transformation is simpler than log-exp maps used in the case of data belonging to Riemannian manifolds and recently extended to consider the pseudo-Riemannian structure of an input space equipped with the Wasserstein distance.