Evaluating the overall sensitivity of saliency-based explanation methods

TL;DR

This paper proposes a formal, model-agnostic methodology to evaluate the sensitivity of saliency-based explanation methods for deep learning models, aiming to improve faithfulness assessments across domains.

Contribution

It extends an existing sensitivity test with formal thresholds and criteria, enabling comprehensive comparison of explanation methods for CNNs and beyond.

Findings

Extended the sensitivity test with formal thresholds

Compared multiple explanation methods for CNNs

Discussed the relationship between sensitivity and faithfulness

Abstract

We address the need to generate faithful explanations of "black box" Deep Learning models. Several tests have been proposed to determine aspects of faithfulness of explanation methods, but they lack cross-domain applicability and a rigorous methodology. Hence, we select an existing test that is model agnostic and is well-suited for comparing one aspect of faithfulness (i.e., sensitivity) of multiple explanation methods, and extend it by specifying formal thresh-olds and building criteria to determine the over-all sensitivity of the explanation method. We present examples of how multiple explanation methods for Convolutional Neural Networks can be compared using this extended methodology. Finally, we discuss the relationship between sensitivity and faithfulness and consider how the test can be adapted to assess different explanation methods in other domains.