A Uniform Concentration Inequality for Kernel-Based Two-Sample Statistics

TL;DR

This paper introduces a unified framework for kernel-based two-sample statistics and establishes a new uniform concentration inequality, providing finite-sample and asymptotic error bounds for various distribution discrepancy measures used in machine learning.

Contribution

It presents a novel uniform concentration inequality applicable to multiple kernel-based two-sample metrics, unifying their analysis and enabling performance guarantees in diverse applications.

Findings

Provides upper bounds for estimation errors in distribution discrepancy measures.

Facilitates error analysis for MMD, dCov, HSIC, and related methods.

Enables finite-sample and asymptotic guarantees for statistical procedures.

Abstract

In many contemporary statistical and machine learning methods, one needs to optimize an objective function that depends on the discrepancy between two probability distributions. The discrepancy can be referred to as a metric for distributions. Widely adopted examples of such a metric include Energy Distance (ED), distance Covariance (dCov), Maximum Mean Discrepancy (MMD), and the Hilbert-Schmidt Independence Criterion (HSIC). We show that these metrics can be unified under a general framework of kernel-based two-sample statistics. This paper establishes a novel uniform concentration inequality for the aforementioned kernel-based statistics. Our results provide upper bounds for estimation errors in the associated optimization problems, thereby offering both finite-sample and asymptotic performance guarantees. As illustrative applications, we demonstrate how these bounds facilitate the derivation of error bounds for procedures such as distance covariance-based dimension reduction, distance covariance-based independent component analysis, MMD-based fairness-constrained inference, MMD-based generative model search, and MMD-based generative adversarial networks.