A K-Means, Ward and DBSCAN repeatability study

TL;DR

This study investigates the repeatability of popular clustering algorithms like K-Means, Ward, and DBSCAN, highlighting issues with reproducibility in K-Means when using multiple threads and emphasizing the importance of reproducibility for scientific integrity.

Contribution

The paper decomposes clustering algorithms to identify conditions for repeatability and demonstrates inconsistencies in K-Means results related to threading, encouraging further investigation.

Findings

K-Means results become inconsistent with more than two OpenMP threads

Decomposition reveals stages where repeatability can be compromised

Highlights need for reproducibility standards in clustering algorithms

Abstract

Reproducibility is essential in machine learning because it ensures that a model or experiment yields the same scientific conclusion. For specific algorithms repeatability with bitwise identical results is also a key for scientific integrity because it allows debugging. We decomposed several very popular clustering algorithms: K-Means, DBSCAN and Ward into their fundamental steps, and we identify the conditions required to achieve repeatability at each stage. We use an implementation example with the Python library scikit-learn to examine the repeatable aspects of each method. Our results reveal inconsistent results with K-Means when the number of OpenMP threads exceeds two. This work aims to raise awareness of this issue among both users and developers, encouraging further investigation and potential fixes.