Pointwise Metrics for Clustering Evaluation

TL;DR

This paper introduces pointwise clustering metrics that evaluate the similarity between clusterings, considering item importance and providing detailed insights into clustering quality and errors.

Contribution

It presents a new set of simple, set-theoretic pointwise metrics for clustering evaluation that account for item importance and enable detailed analysis.

Findings

Metrics are easy to understand and mathematically well-behaved.

They can evaluate clusterings at multiple levels, including items, clusters, and slices.

Metrics facilitate in-depth analysis of clustering mistakes and insights.

Abstract

This paper defines pointwise clustering metrics, a collection of metrics for characterizing the similarity of two clusterings. These metrics have several interesting properties which make them attractive for practical applications. They can take into account the relative importance of the various items that are clustered. The metric definitions are based on standard set-theoretic notions and are simple to understand. They characterize aspects that are important for typical applications, such as cluster homogeneity and completeness. It is possible to assign metrics to individual items, clusters, arbitrary slices of items, and the overall clustering. The metrics can provide deep insights, for example they can facilitate drilling deeper into clustering mistakes to understand where they happened, or help to explore slices of items to understand how they were affected. Since the pointwise metrics are mathematically well-behaved, they can provide a strong foundation for a variety of clustering evaluation techniques. In this paper we discuss in depth how the pointwise metrics can be used to evaluate an actual clustering with respect to a ground truth clustering.