Do Metrics for Counterfactual Explanations Align with User Perception?

TL;DR

This study empirically evaluates whether common algorithmic metrics for counterfactual explanations align with human perceptions of explanation quality, revealing weak correlations and dataset dependency, and highlighting the need for human-centered evaluation methods.

Contribution

It provides an empirical comparison between algorithmic metrics and human judgments, demonstrating the limitations of current metrics in capturing user-perceived explanation quality.

Findings

Weak correlation between metrics and human ratings

Dataset-dependent variability in metric effectiveness

Adding more metrics does not reliably improve prediction of human judgments

Abstract

Explainability is widely regarded as essential for trustworthy artificial intelligence systems. However, the metrics commonly used to evaluate counterfactual explanations are algorithmic evaluation metrics that are rarely validated against human judgments of explanation quality. This raises the question of whether such metrics meaningfully reflect user perceptions. We address this question through an empirical study that directly compares algorithmic evaluation metrics with human judgments across three datasets. Participants rated counterfactual explanations along multiple dimensions of perceived quality, which we relate to a comprehensive set of standard counterfactual metrics. We analyze both individual relationships and the extent to which combinations of metrics can predict human assessments. Our results show that correlations between algorithmic metrics and human ratings are generally weak and strongly dataset-dependent. Moreover, increasing the number of metrics used in predictive models does not lead to reliable improvements, indicating structural limitations in how current metrics capture criteria relevant for humans. Overall, our findings suggest that widely used counterfactual evaluation metrics fail to reflect key aspects of explanation quality as perceived by users, underscoring the need for more human-centered approaches to evaluating explainable artificial intelligence.