A Pragmatic Method for Comparing Clusterings with Overlaps and Outliers

TL;DR

This paper introduces a new similarity measure for comparing clustering results that can handle overlaps and outliers, addressing limitations of existing methods and reducing common biases.

Contribution

The paper proposes a pragmatic similarity measure for overlapping and outlier-inclusive clusterings, with demonstrated desirable properties and bias reduction.

Findings

The measure effectively compares clusterings with overlaps and outliers.

It exhibits several desirable mathematical properties.

Experimental results show reduced bias compared to existing measures.

Abstract

Clustering algorithms are an essential part of the unsupervised data science ecosystem, and extrinsic evaluation of clustering algorithms requires a method for comparing the detected clustering to a ground truth clustering. In a general setting, the detected and ground truth clusterings may have outliers (objects belonging to no cluster), overlapping clusters (objects may belong to more than one cluster), or both, but methods for comparing these clusterings are currently undeveloped. In this note, we define a pragmatic similarity measure for comparing clusterings with overlaps and outliers, show that it has several desirable properties, and experimentally confirm that it is not subject to several common biases afflicting other clustering comparison measures.