Random Projections for $k$-means Clustering

TL;DR

This paper demonstrates that random projections can efficiently reduce the dimensionality of data for $k$-means clustering while approximately preserving the optimal clustering, supported by theoretical proofs and empirical validation.

Contribution

The paper introduces a method using random projections to reduce dimensions for $k$-means clustering, with provable guarantees on preserving the optimal partition within a factor of $2+ ext{eps}$.

Findings

Projection dimension $t = ext{Omega}(k / ext{eps}^2)$ suffices.

Algorithm runs in $O(n d ext{eps}^{-2} k / ext{log}(d))$ time.

Experiments confirm speed and accuracy on large datasets.

Abstract

This paper discusses the topic of dimensionality reduction for $k$-means clustering. We prove that any set of $n$ points in $d$ dimensions (rows in a matrix $A \in \RR^{n \times d}$) can be projected into $t = \Omega(k / \eps^2)$ dimensions, for any $\eps \in (0,1/3)$, in $O(n d \lceil \eps^{-2} k/ \log(d) \rceil )$ time, such that with constant probability the optimal $k$-partition of the point set is preserved within a factor of $2+\eps$. The projection is done by post-multiplying $A$ with a $d \times t$ random matrix $R$ having entries $+1/\sqrt{t}$ or $-1/\sqrt{t}$ with equal probability. A numerical implementation of our technique and experiments on a large face images dataset verify the speed and the accuracy of our theoretical results.