A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings

TL;DR

This paper introduces a measure-theoretic framework for kernel conditional mean embeddings, overcoming limitations of operator-based methods, and provides theoretical guarantees and new conditional dependence measures.

Contribution

It develops a measure-theoretic approach to CMEs, enabling rigorous analysis and empirical estimation without restrictive assumptions.

Findings

Established universal consistency of the proposed estimator

Derived conditional maximum mean discrepancy and Hilbert-Schmidt independence criterion

Demonstrated the methods' behavior through simulations

Abstract

We present an operator-free, measure-theoretic approach to the conditional mean embedding (CME) as a random variable taking values in a reproducing kernel Hilbert space. While the kernel mean embedding of unconditional distributions has been defined rigorously, the existing operator-based approach of the conditional version depends on stringent assumptions that hinder its analysis. We overcome this limitation via a measure-theoretic treatment of CMEs. We derive a natural regression interpretation to obtain empirical estimates, and provide a thorough theoretical analysis thereof, including universal consistency. As natural by-products, we obtain the conditional analogues of the maximum mean discrepancy and Hilbert-Schmidt independence criterion, and demonstrate their behaviour via simulations.