Normalised clustering accuracy: An asymmetric external cluster validity measure

TL;DR

This paper introduces a normalized, asymmetric external cluster validity measure called normalised clustering accuracy, addressing limitations of classical similarity scores by improving interpretability and robustness in evaluating clustering algorithms.

Contribution

The paper proposes a novel external validity measure that corrects for imbalanced cluster sizes and offers better interpretability compared to traditional scores.

Findings

The new measure is scale-invariant and monotonic.

It corrects for imbalanced cluster sizes.

It better identifies worst-case scenarios.

Abstract

There is no, nor will there ever be, single best clustering algorithm. Nevertheless, we would still like to be able to distinguish between methods that work well on certain task types and those that systematically underperform. Clustering algorithms are traditionally evaluated using either internal or external validity measures. Internal measures quantify different aspects of the obtained partitions, e.g., the average degree of cluster compactness or point separability. However, their validity is questionable because the clusterings they endorse can sometimes be meaningless. External measures, on the other hand, compare the algorithms' outputs to fixed ground truth groupings provided by experts. In this paper, we argue that the commonly used classical partition similarity scores, such as the normalised mutual information, Fowlkes-Mallows, or adjusted Rand index, miss some desirable properties. In particular, they do not identify worst-case scenarios correctly, nor are they easily interpretable. As a consequence, the evaluation of clustering algorithms on diverse benchmark datasets can be difficult. To remedy these issues, we propose and analyse a new measure: a version of the optimal set-matching accuracy, which is normalised, monotonic with respect to some similarity relation, scale-invariant, and corrected for the imbalancedness of cluster sizes (but neither symmetric nor adjusted for chance).