Associativity-Peakiness Metric for Contingency Tables

TL;DR

This paper introduces the Associativity Peakiness (AP) metric, a new performance measure for contingency tables from clustering algorithms, demonstrating its higher dynamic range and efficiency over existing metrics.

Contribution

The paper proposes the AP metric for evaluating clustering performance from contingency tables, filling a gap not addressed by existing vector-based metrics.

Findings

AP metric has higher dynamic range than existing metrics.

AP is more computationally efficient than comparable metrics.

Simulations with 500 contingency tables validate AP's effectiveness.

Abstract

For the use case of comparing the performance of clustering algorithms whose output is a contingency table, a single performance metric for contingency tables is needed. Such a metric is vital for comparative performance analysis of clustering algorithms. A survey of publicly available literature did not show the presence of such a metric. Metrics do exist for vector pairs of truth values and predicted values, which are an alternative form of output of clustering algorithms. However, the metrics for vector pairs do not reveal the presence of detailed features that are apparent in contingency tables. This paper presents the Associativity Peakiness (AP) metric, which characterizes aspects of clustering algorithm performance that are critical for predicting a clustering algorithm's performance when deployed. The AP metric is analogous to measures of quality for confusion matrices that are outputs of supervised learning algorithms. This paper presents results from simulations in which 500 contingency tables were generated for multiple test scenarios. The results show that for the use case of evaluating clustering algorithms, the AP metric characterizes performance of contingency tables with higher dynamic range than publicly available metrics, and that it is computationally more efficient than comparable publicly available metrics.