Breathing K-Means: Superior K-Means Solutions through Dynamic K-Values

TL;DR

Breathing K-Means introduces a dynamic approach to optimize clustering solutions by cyclically adjusting the number of centroids, outperforming traditional methods in solution quality and speed.

Contribution

The paper presents the breathing k-means algorithm, a novel method that adaptively modifies the number of centroids to improve clustering results over standard k-means++.

Findings

Breathing k-means consistently outperforms greedy k-means++ in solution quality.

Breathing k-means demonstrates faster convergence compared to baseline algorithms.

The method maintains superior performance even with fewer runs, saving computational resources.

Abstract

We introduce the breathing k-means algorithm, which on average significantly improves solutions obtained by the widely-known greedy k-means++ algorithm, the default method for k-means clustering in the scikit-learn package. The improvements are achieved through a novel ``breathing'' technique, that cyclically increases and decreases the number of centroids based on local error and utility measures. We conducted experiments using greedy k-means++ as a baseline, comparing it with breathing k-means and five other k-means algorithms. Among the methods investigated, only breathing k-means and better k-means++ consistently outperformed the baseline, with breathing k-means demonstrating a substantial lead. This superior performance was maintained even when comparing the best result of ten runs for all other algorithms to a single run of breathing k-means, highlighting its effectiveness and speed. Our findings indicate that the breathing k-means algorithm outperforms the other k-means techniques, especially greedy k-means++ with ten repetitions, which it dominates in both solution quality and speed. This positions breathing k-means (with the built-in initialization by a single run of greedy k-means++) as a superior alternative to running greedy k-means++ on its own.