A Simple Saliency Method That Passes the Sanity Checks
Arushi Gupta, Sanjeev Arora

TL;DR
This paper introduces a simple, efficient method called 'Competitive Gradient $owtie$ Input' that enhances saliency maps, enabling them to pass sanity checks and better reflect true model decision features.
Contribution
The authors propose a straightforward fix involving competition among label-specific saliency maps, improving the validity of attribution methods without extra training.
Findings
CGI method passes sanity checks effectively
Requires only input and gradient, no additional training
Theoretically justified for ReLU networks
Abstract
There is great interest in "saliency methods" (also called "attribution methods"), which give "explanations" for a deep net's decision, by assigning a "score" to each feature/pixel in the input. Their design usually involves credit-assignment via the gradient of the output with respect to input. Recently Adebayo et al. [arXiv:1810.03292] questioned the validity of many of these methods since they do not pass simple *sanity checks* which test whether the scores shift/vanish when layers of the trained net are randomized, or when the net is retrained using random labels for inputs. We propose a simple fix to existing saliency methods that helps them pass sanity checks, which we call "competition for pixels". This involves computing saliency maps for all possible labels in the classification task, and using a simple competition among them to identify and remove less relevant pixels from…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Adversarial Robustness in Machine Learning · Advanced Neural Network Applications
Methods*Communicated@Fast*How Do I Communicate to Expedia?
