Variance-Aware Estimation of Kernel Mean Embedding

TL;DR

This paper introduces a variance-aware approach to kernel mean embeddings that accelerates convergence rates by leveraging variance information, even when unknown, and extends these results to dependent data for improved statistical tasks.

Contribution

It proposes a novel variance-aware estimation method for kernel mean embeddings that accelerates convergence and adapts to unknown variance, extending to dependent data.

Findings

Variance-aware estimation improves convergence speed.

Data-driven variance estimation is effective.

Method applies to stationary mixing sequences.

Abstract

An important feature of kernel mean embeddings (KME) is that the rate of convergence of the empirical KME to the true distribution KME can be bounded independently of the dimension of the space, properties of the distribution and smoothness features of the kernel. We show how to speed-up convergence by leveraging variance information in the reproducing kernel Hilbert space. Furthermore, we show that even when such information is a priori unknown, we can efficiently estimate it from the data, recovering the desiderata of a distribution agnostic bound that enjoys acceleration in fortuitous settings. We further extend our results from independent data to stationary mixing sequences and illustrate our methods in the context of hypothesis testing and robust parametric estimation.