Sensitivity Sampling for $k$-Means: Worst Case and Stability Optimal Coreset Bounds

TL;DR

This paper demonstrates that Sensitivity Sampling produces optimal coresets for $k$-means, especially on well-clusterable data, and extends these results to $k$-median and general metric spaces, improving efficiency and understanding.

Contribution

It proves that Sensitivity Sampling yields size-optimal coresets for worst-case and well-clusterable data, and extends these bounds to broader clustering problems and metric spaces.

Findings

Sensitivity Sampling achieves optimal coreset sizes for worst-case $k$-means.

For well-clusterable data, coresets are significantly smaller, size $ ilde{O}(k/ ext{epsilon}^2)$.

Coreset size lower bounds match the upper bounds for stable instances.

Abstract

Coresets are arguably the most popular compression paradigm for center-based clustering objectives such as $k$-means. Given a point set $P$, a coreset $\Omega$ is a small, weighted summary that preserves the cost of all candidate solutions $S$ up to a $(1\pm \varepsilon)$ factor. For $k$-means in $d$-dimensional Euclidean space the cost for solution $S$ is $\sum_{p\in P}\min_{s\in S}\|p-s\|^2$. A very popular method for coreset construction, both in theory and practice, is Sensitivity Sampling, where points are sampled in proportion to their importance. We show that Sensitivity Sampling yields optimal coresets of size $\tilde{O}(k/\varepsilon^2\min(\sqrt{k},\varepsilon^{-2}))$ for worst-case instances. Uniquely among all known coreset algorithms, for well-clusterable data sets with $\Omega(1)$ cost stability, Sensitivity Sampling gives coresets of size $\tilde{O}(k/\varepsilon^2)$, improving over the worst-case lower bound. Notably, Sensitivity Sampling does not have to know the cost stability in order to exploit it: It is appropriately sensitive to the clusterability of the data set while being oblivious to it. We also show that any coreset for stable instances consisting of only input points must have size $\Omega(k/\varepsilon^2)$. Our results for Sensitivity Sampling also extend to the $k$-median problem, and more general metric spaces.