Characteristic and Universal Tensor Product Kernels

TL;DR

This paper investigates the theoretical properties of tensor product kernels used in maximum mean discrepancy and Hilbert-Schmidt independence criterion, focusing on their ability to characterize independence and discriminate distributions.

Contribution

It provides a theoretical analysis of when tensor product kernels are characteristic, addressing gaps in understanding their discriminative power.

Findings

Analyzes conditions under which HSIC characterizes independence

Identifies when MMD with tensor product kernels can distinguish distributions

Provides theoretical foundations for tensor product kernel properties

Abstract

Maximum mean discrepancy (MMD), also called energy distance or N-distance in statistics and Hilbert-Schmidt independence criterion (HSIC), specifically distance covariance in statistics, are among the most popular and successful approaches to quantify the difference and independence of random variables, respectively. Thanks to their kernel-based foundations, MMD and HSIC are applicable on a wide variety of domains. Despite their tremendous success, quite little is known about when HSIC characterizes independence and when MMD with tensor product kernel can discriminate probability distributions. In this paper, we answer these questions by studying various notions of characteristic property of the tensor product kernel.