Convergence of online $k$-means

TL;DR

This paper proves that online $k$-means algorithms converge asymptotically to stationary points when performed on streaming data, by interpreting them as stochastic gradient descent with adaptive learning rates.

Contribution

It establishes the convergence of a broad class of online $k$-means algorithms by linking them to stochastic gradient descent and extending existing optimization techniques.

Findings

Centers converge to stationary points asymptotically

Online $k$-means can be viewed as stochastic gradient descent

Convergence holds under adaptive, center-dependent learning rates

Abstract

We prove asymptotic convergence for a general class of $k$-means algorithms performed over streaming data from a distribution: the centers asymptotically converge to the set of stationary points of the $k$-means cost function. To do so, we show that online $k$-means over a distribution can be interpreted as stochastic gradient descent with a stochastic learning rate schedule. Then, we prove convergence by extending techniques used in optimization literature to handle settings where center-specific learning rates may depend on the past trajectory of the centers.