On the Robustness of Interpretability Methods

TL;DR

This paper emphasizes the importance of robustness in interpretability methods, introduces metrics to measure it, reveals current methods' shortcomings, and suggests ways to improve robustness in explanations.

Contribution

The paper introduces new metrics for robustness, evaluates existing interpretability methods, and proposes techniques to enhance their robustness.

Findings

Current interpretability methods lack robustness according to proposed metrics.

Robustness of explanations can be quantitatively assessed with new metrics.

Proposed methods can improve the robustness of existing interpretability approaches.

Abstract

We argue that robustness of explanations---i.e., that similar inputs should give rise to similar explanations---is a key desideratum for interpretability. We introduce metrics to quantify robustness and demonstrate that current methods do not perform well according to these metrics. Finally, we propose ways that robustness can be enforced on existing interpretability approaches.