Explaining Black-Box Clustering Pipelines With Cluster-Explorer

TL;DR

Cluster-Explorer is a novel tool that explains black-box clustering results by identifying concise predicate conjunctions, improving interpretability and efficiency over existing XAI methods.

Contribution

It introduces a new approach that formulates cluster explanations as frequent-itemsets mining, enabling effective and efficient explanations for black-box clustering pipelines.

Findings

Outperforms XAI baselines in explanation quality

Reduces computational costs through attribute selection

Demonstrates effectiveness on 98 clustering results and user study

Abstract

Explaining the results of clustering pipelines by unraveling the characteristics of each cluster is a challenging task, often addressed manually through visualizations and queries. Existing solutions from the domain of Explainable Artificial Intelligence (XAI) are largely ineffective for cluster explanations, and interpretable-by-design clustering algorithms may be unsuitable when the clustering algorithm does not fit the data properties. To bridge this gap, we introduce Cluster-Explorer, a novel explainability tool for black-box clustering pipelines. Our approach formulates the explanation of clusters as the identification of concise conjunctions of predicates that maximize the coverage of the cluster's data points while minimizing separation from other clusters. We achieve this by reducing the problem to generalized frequent-itemsets mining (gFIM), where items correspond to explanation predicates, and itemset frequency indicates coverage. To enhance efficiency, we leverage inherent problem properties and implement attribute selection to further reduce computational costs. Experimental evaluations on a benchmark collection of 98 clustering results, as well as a user study, demonstrate the superiority of Cluster-Explorer in both explanation quality and execution times compared to XAI baselines.