Kernelized Cumulants: Beyond Kernel Mean Embeddings

TL;DR

This paper introduces kernelized cumulants in RKHS, extending classical cumulants to provide versatile, low-variance statistics that enhance data analysis with minimal computational overhead.

Contribution

It develops a novel framework for kernelized cumulants using tensor algebra, generalizing existing statistics like MMD and HSIC, with theoretical and empirical validation.

Findings

Kernelized cumulants are computationally tractable via kernel tricks.

Going beyond degree one offers advantages in statistical analysis.

Empirical results show improved performance on synthetic and real data.

Abstract

In $\mathbb R^d$, it is well-known that cumulants provide an alternative to moments that can achieve the same goals with numerous benefits such as lower variance estimators. In this paper we extend cumulants to reproducing kernel Hilbert spaces (RKHS) using tools from tensor algebras and show that they are computationally tractable by a kernel trick. These kernelized cumulants provide a new set of all-purpose statistics; the classical maximum mean discrepancy and Hilbert-Schmidt independence criterion arise as the degree one objects in our general construction. We argue both theoretically and empirically (on synthetic, environmental, and traffic data analysis) that going beyond degree one has several advantages and can be achieved with the same computational complexity and minimal overhead in our experiments.