$K-$means with learned metrics

TL;DR

This paper develops a unified theoretical framework for the consistency of $k$-means clustering when both the measure and the metric are unknown and estimated, with applications to various metric learning methods.

Contribution

It proves continuity and stability of $k$-means in the measured Gromov-Hausdorff topology, enabling new consistency results for several metric learning estimators.

Findings

Established stability of $k$-means with respect to measured Gromov-Hausdorff topology.

Proved consistency for $k$-means based on Isomap, diffusion, and Wasserstein distances.

Extended results to applications like first passage percolation and discrete length space approximations.

Abstract

We study the Fr\'echet $k-$means of a metric measure space when both the measure and the distance are unknown and have to be estimated. We prove a general result that states that the $k-$means are continuous with respect to the measured Gromov-Hausdorff topology. In this situation, we also prove a stability result for the Voronoi clusters they determine. We do not assume uniqueness of the set of $k-$means, but when it is unique, the results are stronger. This framework provides a unified approach to proving consistency for a wide range of metric learning procedures. As concrete applications, we obtain new consistency results for several important estimators that were previously unestablished, even when $k=1$. These include $k-$means based on: (i) Isomap and Fermat geodesic distances on manifolds, (ii) difussion distances, (iii) Wasserstein distances computed with respect to learned ground metrics. Finally, we consider applications beyond the statistical inference paradigm like (iv) first passage percolation and (v) discrete approximations of length spaces.