Fast $k$-means Seeding Under The Manifold Hypothesis

TL;DR

This paper introduces a new seeding method for $k$-means clustering, leveraging the manifold hypothesis to achieve faster runtimes with predictable approximation guarantees, validated through extensive empirical testing.

Contribution

It proposes $ ext{Qkmeans}$, a novel seeding algorithm that exploits geometric properties of data on low-dimensional manifolds for improved efficiency and theoretical guarantees.

Findings

$ ext{Qkmeans}$ achieves $O( ho^{-2} ext{log} k)$ approximation.

The algorithm runs in $O(nD) + ilde{O}( ext{epsilon}^{1+ ho} ho^{-1}k^{1+ ext{gamma}})$ time.

Empirical results validate theoretical predictions across various domains.

Abstract

We study beyond worst case analysis for the $k$-means problem where the goal is to model typical instances of $k$-means arising in practice. Existing theoretical approaches provide guarantees under certain assumptions on the optimal solutions to $k$-means, making them difficult to validate in practice. We propose the manifold hypothesis, where data obtained in ambient dimension $D$ concentrates around a low dimensional manifold of intrinsic dimension $d$, as a reasonable assumption to model real world clustering instances. We identify key geometric properties of datasets which have theoretically predictable scaling laws depending on the quantization exponent $\varepsilon = 2/d$ using techniques from optimum quantization theory. We show how to exploit these regularities to design a fast seeding method called $\operatorname{Qkmeans}$ which provides $O(\rho^{-2} \log k)$ approximate solutions to the $k$-means problem in time $O(nD) + \widetilde{O}(\varepsilon^{1+\rho}\rho^{-1}k^{1+\gamma})$; where the exponent $\gamma = \varepsilon + \rho$ for an input parameter $\rho < 1$. This allows us to obtain new runtime - quality tradeoffs. We perform a large scale empirical study across various domains to validate our theoretical predictions and algorithm performance to bridge theory and practice for beyond worst case data clustering.