Learning Quantifiable Visual Explanations Without Ground-Truth

TL;DR

This paper introduces a new quantifiable metric for evaluating XAI methods based on input perturbation, and proposes a novel explanation technique that improves explanation quality without affecting model performance.

Contribution

It presents a formal metric for XAI evaluation and a new explanation method trained with this metric, outperforming existing techniques.

Findings

The proposed metric aligns better with human intuition than existing metrics.

The new explanation method produces more accurate causal explanations.

Explanations do not degrade the underlying model's performance.

Abstract

Explainable AI (XAI) techniques are increasingly important for the validation and responsible use of modern deep learning models, but are difficult to evaluate due to the lack of good ground-truth to compare against. We propose a framework that serves as a quantifiable metric for the quality of XAI methods, based on continuous input perturbation. Our metric formally considers the sufficiency and necessity of the attributed information to the model's decision-making, and we illustrate a range of cases where it aligns better with human intuitions of explanation quality than do existing metrics. To exploit the properties of this metric, we also propose a novel XAI method, considering the case where we fine-tune a model using a differentiable approximation of the metric as a supervision signal. The result is an adapter module that can be trained on top of any black-box model to output causal explanations of the model's decision process, without degrading model performance. We show that the explanations generated by this method outperform those of competing XAI techniques according to a number of quantifiable metrics.