Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D

TL;DR

This paper reviews and improves exact polynomial-time algorithms for 1D k-Means and related clustering problems, extending them to Bregman divergences and optimizing space and reporting efficiency.

Contribution

It uncovers overlooked historical algorithms, compares their theoretical properties, and introduces space-efficient data structures for fast exact clustering in 1D.

Findings

Existing algorithms can be optimized for space and speed.

Algorithms generalize to Bregman divergences and absolute distances.

Practical experiments demonstrate performance improvements.

Abstract

The $k$-Means clustering problem on $n$ points is NP-Hard for any dimension $d\ge 2$, however, for the 1D case there exists exact polynomial time algorithms. Previous literature reported an $O(kn^2)$ time dynamic programming algorithm that uses $O(kn)$ space. It turns out that the problem has been considered under a different name more than twenty years ago. We present all the existing work that had been overlooked and compare the various solutions theoretically. Moreover, we show how to reduce the space usage for some of them, as well as generalize them to data structures that can quickly report an optimal $k$-Means clustering for any $k$. Finally we also generalize all the algorithms to work for the absolute distance and to work for any Bregman Divergence. We complement our theoretical contributions by experiments that compare the practical performance of the various algorithms.