The Blame Problem in Evaluating Local Explanations, and How to Tackle it

TL;DR

This paper introduces a new taxonomy for evaluating local explanations, highlights the prevalent 'blame problem' in existing methods, and advocates for ground-truth-based evaluation as a more reliable approach, though challenges remain.

Contribution

It proposes a comprehensive taxonomy for local explanation evaluation and identifies the 'blame problem' as a key issue, emphasizing ground-truth-based evaluation as a promising solution.

Findings

Most evaluation methods suffer from the 'blame problem'

Ground-truth-based evaluation is more reasonable

Evaluation of local explanations remains an open challenge

Abstract

The number of local model-agnostic explanation techniques proposed has grown rapidly recently. One main reason is that the bar for developing new explainability techniques is low due to the lack of optimal evaluation measures. Without rigorous measures, it is hard to have concrete evidence of whether the new explanation techniques can significantly outperform their predecessors. Our study proposes a new taxonomy for evaluating local explanations: robustness, evaluation using ground truth from synthetic datasets and interpretable models, model randomization, and human-grounded evaluation. Using this proposed taxonomy, we highlight that all categories of evaluation methods, except those based on the ground truth from interpretable models, suffer from a problem we call the "blame problem." In our study, we argue that this category of evaluation measure is a more reasonable method for evaluating local model-agnostic explanations. However, we show that even this category of evaluation measures has further limitations. The evaluation of local explanations remains an open research problem.