Gaussian processes with multidimensional distribution inputs via optimal transport and Hilbertian embedding

TL;DR

This paper develops a novel approach for Gaussian processes with multidimensional distribution inputs using optimal transport and Hilbert space embeddings, addressing limitations of Wasserstein-based kernels in higher dimensions.

Contribution

It introduces a new positive definite kernel construction for multivariate distributions via Hilbert space embeddings and analyzes their statistical and practical properties.

Findings

Kernel constructions are valid in high-dimensional spaces.

The proposed kernels are consistent with empirical barycenters.

Application to regression shows promising results in simulations.

Abstract

In this work, we investigate Gaussian Processes indexed by multidimensional distributions. While directly constructing radial positive definite kernels based on the Wasserstein distance has been proven to be possible in the unidimensional case, such constructions do not extend well to the multidimensional case as we illustrate here. To tackle the problem of defining positive definite kernels between multivariate distributions based on optimal transport, we appeal instead to Hilbert space embeddings relying on optimal transport maps to a reference distribution, that we suggest to take as a Wasserstein barycenter. We characterize in turn radial positive definite kernels on Hilbert spaces, and show that the covariance parameters of virtually all parametric families of covariance functions are microergodic in the case of (infinite-dimensional) Hilbert spaces. We also investigate statistical properties of our suggested positive definite kernels on multidimensional distributions, with a focus on consistency when a population Wasserstein barycenter is replaced by an empirical barycenter and additional explicit results in the special case of Gaussian distributions. Finally, we study the Gaussian process methodology based on our suggested positive definite kernels in regression problems with multidimensional distribution inputs, on simulation data stemming both from synthetic examples and from a mechanical engineering test case.