An empirical comparison between stochastic and deterministic centroid initialisation for K-Means variations

TL;DR

This paper compares stochastic and deterministic centroid initialization methods for K-Means, showing that deterministic methods generally perform better and are more efficient, though stochastic methods can sometimes yield better results with multiple runs.

Contribution

It provides an extensive empirical benchmark comparing various initialization techniques for K-Means, highlighting the advantages of deterministic methods over stochastic ones.

Findings

Deterministic initializations outperform stochastic methods on average.

Multiple runs of stochastic methods can improve clustering quality.

Deterministic methods are more time-efficient and competitive.

Abstract

K-Means is one of the most used algorithms for data clustering and the usual clustering method for benchmarking. Despite its wide application it is well-known that it suffers from a series of disadvantages; it is only able to find local minima and the positions of the initial clustering centres (centroids) can greatly affect the clustering solution. Over the years many K-Means variations and initialisation techniques have been proposed with different degrees of complexity. In this study we focus on common K-Means variations along with a range of deterministic and stochastic initialisation techniques. We show that, on average, more sophisticated initialisation techniques alleviate the need for complex clustering methods. Furthermore, deterministic methods perform better than stochastic methods. However, there is a trade-off: less sophisticated stochastic methods, executed multiple times, can result in better clustering. Factoring in execution time, deterministic methods can be competitive and result in a good clustering solution. These conclusions are obtained through extensive benchmarking using a range of synthetic model generators and real-world data sets.