Likelihood Ratio Tests by Kernel Gaussian Embedding

TL;DR

This paper introduces a kernel-based nonparametric two-sample test using Gaussian embeddings and likelihood ratios, achieving high power in high-dimensional settings through a regularized, permutation-calibrated approach.

Contribution

It develops a novel likelihood ratio test based on kernel Gaussian embeddings, extending previous MMD methods with improved power and theoretical guarantees.

Findings

The test is consistent and has uniform power guarantees.

Empirical results show significant power gains over existing methods.

Effective in high-dimensional and weak-signal scenarios.

Abstract

We propose a novel kernel-based nonparametric two-sample test, employing the combined use of kernel mean and kernel covariance embedding. Our test builds on recent results showing how such combined embeddings map distinct probability measures to mutually singular Gaussian measures on the kernel's RKHS. Leveraging this ``separation of measure phenomenon", we construct a test statistic based on the relative entropy between the Gaussian embeddings, in effect the likelihood ratio. The likelihood ratio is specifically tailored to detect equality versus singularity of two Gaussians, and satisfies a ``$0/\infty$" law, in that it vanishes under the null and diverges under the alternative. To implement the test in finite samples, we introduce a regularised version, calibrated by way of permutation. We prove consistency, establish uniform power guarantees under mild conditions, and discuss how our framework unifies and extends prior approaches based on spectrally regularized MMD. Empirical results on synthetic and real data demonstrate remarkable gains in power compared to state-of-the-art methods, particularly in high-dimensional and weak-signal regimes.