The linear conditional expectation in Hilbert space

TL;DR

This paper analyzes the properties of linear conditional expectation in infinite-dimensional Hilbert spaces, introduces a regularization method, and provides a new derivation of the conditional mean embedding used in machine learning.

Contribution

It establishes the analytical properties of LCE in Hilbert spaces and introduces a regularization procedure within the space of affine Hilbert--Schmidt operators.

Findings

Analytical properties of LCE in Hilbert spaces are characterized.

A regularization procedure for LCE is developed.

A new derivation of the conditional mean embedding formula is provided.

Abstract

The linear conditional expectation (LCE) provides a best linear (or rather, affine) estimate of the conditional expectation and hence plays an important r\^ole in approximate Bayesian inference, especially the Bayes linear approach. This article establishes the analytical properties of the LCE in an infinite-dimensional Hilbert space context. In addition, working in the space of affine Hilbert--Schmidt operators, we establish a regularisation procedure for this LCE. As an important application, we obtain a simple alternative derivation and intuitive justification of the conditional mean embedding formula, a concept widely used in machine learning to perform the conditioning of random variables by embedding them into reproducing kernel Hilbert spaces.