Multi-Swap $k$-Means++

TL;DR

This paper introduces a generalized multi-swap local search algorithm for $k$-means clustering that achieves a near-optimal approximation ratio and demonstrates significant practical improvements over previous methods.

Contribution

It extends existing local search algorithms by allowing multiple simultaneous swaps, achieving a $9 + \\varepsilon$ approximation ratio, the best possible for local search, with practical performance gains.

Findings

Achieves a $9 + \\varepsilon$ approximation ratio for $k$-means clustering.

Demonstrates significant quality improvements over previous local search methods.

Shows practical effectiveness on several datasets.

Abstract

The $k$-means++ algorithm of Arthur and Vassilvitskii (SODA 2007) is often the practitioners' choice algorithm for optimizing the popular $k$-means clustering objective and is known to give an $O(\log k)$-approximation in expectation. To obtain higher quality solutions, Lattanzi and Sohler (ICML 2019) proposed augmenting $k$-means++ with $O(k \log \log k)$ local search steps obtained through the $k$-means++ sampling distribution to yield a $c$-approximation to the $k$-means clustering problem, where $c$ is a large absolute constant. Here we generalize and extend their local search algorithm by considering larger and more sophisticated local search neighborhoods hence allowing to swap multiple centers at the same time. Our algorithm achieves a $9 + \varepsilon$ approximation ratio, which is the best possible for local search. Importantly we show that our approach yields substantial practical improvements, we show significant quality improvements over the approach of Lattanzi and Sohler (ICML 2019) on several datasets.