A Vulnerability of Attribution Methods Using Pre-Softmax Scores

TL;DR

This paper reveals a vulnerability in attribution methods for CNN classifiers, showing that small model modifications can significantly affect explanations without changing the model's output.

Contribution

It identifies a new vulnerability where attribution explanations can be manipulated by minor model changes, independent of output alterations.

Findings

Small model modifications can alter attribution explanations

Attribution methods are vulnerable without changing model outputs

Highlights a need for more robust explanation techniques

Abstract

We discuss a vulnerability involving a category of attribution methods used to provide explanations for the outputs of convolutional neural networks working as classifiers. It is known that this type of networks are vulnerable to adversarial attacks, in which imperceptible perturbations of the input may alter the outputs of the model. In contrast, here we focus on effects that small modifications in the model may cause on the attribution method without altering the model outputs.