More Clustering Quality Metrics for ABCDE

TL;DR

This paper extends the ABCDE clustering evaluation framework by introducing new metrics to better characterize clustering differences, including DeltaRecall, IQ, and absolute Precision and Recall, enhancing the assessment of clustering quality improvements.

Contribution

It introduces new metrics such as DeltaRecall and IQ, and methods to evaluate absolute Precision and Recall within the ABCDE framework, advancing clustering quality analysis.

Findings

Introduced a technique to characterize DeltaRecall of clustering changes.

Proposed a new metric called IQ to measure quality improvement.

Outlined methods to assess absolute Precision and Recall for single clusterings.

Abstract

ABCDE is a technique for evaluating clusterings of very large populations of items. Given two clusterings, namely a Baseline clustering and an Experiment clustering, ABCDE can characterize their differences with impact and quality metrics, and thus help to determine which clustering to prefer. We previously described the basic quality metrics of ABCDE, namely the GoodSplitRate, BadSplitRate, GoodMergeRate, BadMergeRate and DeltaPrecision, and how to estimate them on the basis of human judgements. This paper extends that treatment with more quality metrics. It describes a technique that aims to characterize the DeltaRecall of the clustering change. It introduces a new metric, called IQ, to characterize the degree to which the clustering diff translates into an improvement in the quality. Ideally, a large diff would improve the quality by a large amount. Finally, this paper mentions ways to characterize the absolute Precision and Recall of a single clustering with ABCDE.