Recursive Estimation of Conditional Kernel Mean Embeddings

TL;DR

This paper introduces a recursive algorithm for estimating conditional kernel mean embeddings in Hilbert spaces, with proven consistency and demonstrated applications across various geometric domains.

Contribution

It presents a novel recursive estimation method for conditional kernel mean embeddings with theoretical consistency proofs and broad applicability.

Findings

Proven weak and strong $L_2$ consistency of the estimator.

Established asymptotic bounds for convergence.

Validated on Euclidean, Riemannian, and function space inputs.

Abstract

Kernel mean embeddings, a widely used technique in machine learning, map probability distributions to elements of a reproducing kernel Hilbert space (RKHS). For supervised learning problems, where input-output pairs are observed, the conditional distribution of outputs given the inputs is a key object. The input dependent conditional distribution of an output can be encoded with an RKHS valued function, the conditional kernel mean map. In this paper we present a new recursive algorithm to estimate the conditional kernel mean map in a Hilbert space valued $L_2$ space, that is in a Bochner space. We prove the weak and strong $L_2$ consistency of our recursive estimator under mild conditions. The idea is to generalize Stone's theorem for Hilbert space valued regression in a locally compact Polish space. We present new insights about conditional kernel mean embeddings and give strong asymptotic bounds regarding the convergence of the proposed recursive method. Finally, the results are demonstrated on three application domains: for inputs coming from Euclidean spaces, Riemannian manifolds and locally compact subsets of function spaces.