A Rigorous Theory of Conditional Mean Embeddings

TL;DR

This paper develops a rigorous mathematical framework for conditional mean embeddings in RKHSs, clarifying their properties, limitations, and connections to Gaussian conditioning, thereby advancing their theoretical foundation.

Contribution

It provides a comprehensive, rigorous theory for both centred and uncentred CMEs, weakening applicability conditions and linking to Gaussian conditioning in Hilbert spaces.

Findings

Established a rigorous mathematical foundation for CMEs

Compared merits and issues of centred vs uncentred CMEs

Connected CMEs to Gaussian conditioning in Hilbert spaces

Abstract

Conditional mean embeddings (CMEs) have proven themselves to be a powerful tool in many machine learning applications. They allow the efficient conditioning of probability distributions within the corresponding reproducing kernel Hilbert spaces (RKHSs) by providing a linear-algebraic relation for the kernel mean embeddings of the respective joint and conditional probability distributions. Both centred and uncentred covariance operators have been used to define CMEs in the existing literature. In this paper, we develop a mathematically rigorous theory for both variants, discuss the merits and problems of each, and significantly weaken the conditions for applicability of CMEs. In the course of this, we demonstrate a beautiful connection to Gaussian conditioning in Hilbert spaces.