A Kernel Measure of Dissimilarity between $M$ Distributions

TL;DR

This paper introduces a nonparametric kernel-based measure called KMD to quantify dissimilarity among multiple distributions, with properties similar to divergence measures, efficient computation, and a consistent test for equality of distributions.

Contribution

The paper proposes a novel kernel measure of multi-sample dissimilarity with theoretical properties, efficient computation, and a new test for distribution equality.

Findings

KMD ranges between 0 and 1, indicating identical or mutually singular distributions.

The sample KMD can be computed in near linear time using k-nearest neighbor graphs.

The proposed test is consistent and has well-characterized asymptotic properties.

Abstract

Given $M \geq 2$ distributions defined on a general measurable space, we introduce a nonparametric (kernel) measure of multi-sample dissimilarity (KMD) -- a parameter that quantifies the difference between the $M$ distributions. The population KMD, which takes values between 0 and 1, is 0 if and only if all the $M$ distributions are the same, and 1 if and only if all the distributions are mutually singular. Moreover, KMD possesses many properties commonly associated with $f$-divergences such as the data processing inequality and invariance under bijective transformations. The sample estimate of KMD, based on independent observations from the $M$ distributions, can be computed in near linear time (up to logarithmic factors) using $k$-nearest neighbor graphs (for $k \ge 1$ fixed). We develop an easily implementable test for the equality of $M$ distributions based on the sample KMD that is consistent against all alternatives where at least two distributions are not equal. We prove central limit theorems for the sample KMD, and provide a complete characterization of the asymptotic power of the test, as well as its detection threshold. The usefulness of our measure is demonstrated via real and synthetic data examples; our method is also implemented in an R package.