Universal consistency of Wasserstein $k$-NN classifier: Negative and Positive Results

TL;DR

This paper investigates the universal consistency of the Wasserstein $k$-NN classifier, revealing limitations in general spaces and proposing conditions under which it is consistent, including specific structured measure spaces.

Contribution

It demonstrates the non-universality of Wasserstein $k$-NN classifier in general, and establishes its universal consistency on structured measure spaces with certain geometric properties.

Findings

Not universally consistent on all measure spaces supported in (0,1)

Universally consistent on spaces with $\sigma$-finite metric dimension

Consistent on finite, Gaussian, and wavelet-based measure spaces

Abstract

The Wasserstein distance provides a notion of dissimilarities between probability measures, which has recent applications in learning of structured data with varying size such as images and text documents. In this work, we study the $k$-nearest neighbor classifier ($k$-NN) of probability measures under the Wasserstein distance. We show that the $k$-NN classifier is not universally consistent on the space of measures supported in $(0,1)$. As any Euclidean ball contains a copy of $(0,1)$, one should not expect to obtain universal consistency without some restriction on the base metric space, or the Wasserstein space itself. To this end, via the notion of $\sigma$-finite metric dimension, we show that the $k$-NN classifier is universally consistent on spaces of measures supported in a $\sigma$-uniformly discrete set. In addition, by studying the geodesic structures of the Wasserstein spaces for $p=1$ and $p=2$, we show that the $k$-NN classifier is universally consistent on the space of measures supported on a finite set, the space of Gaussian measures, and the space of measures with densities expressed as finite wavelet series.