CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency

TL;DR

CAMERAS is a novel method for generating high-resolution, sanity-preserving saliency maps for deep visual models, enabling better interpretability and robustness analysis without external priors.

Contribution

It introduces a multi-scale fusion approach to produce detailed saliency maps that maintain model sanity and improve interpretability over existing methods.

Findings

High-resolution saliency maps reveal detailed input feature importance.

Saliency-guided attacks reduce adversarial signal strength.

Method offers new evaluation metrics and sanity checks.

Abstract

Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. However, class-insensitivity of the earlier layers in a network only allows saliency computation with low resolution activation maps of the deeper layers, resulting in compromised image saliency. Remedifying this can lead to sanity failures. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors and preserving the map sanity. Our method systematically performs multi-scale accumulation and fusion of the activation maps and backpropagated gradients to compute precise saliency maps. From accurate image saliency to articulation of relative importance of input features for different models, and precise discrimination between model perception of visually similar objects, our high-resolution mapping offers multiple novel insights into the black-box deep visual models, which are presented in the paper. We also demonstrate the utility of our saliency maps in adversarial setup by drastically reducing the norm of attack signals by focusing them on the precise regions identified by our maps. Our method also inspires new evaluation metrics and a sanity check for this developing research direction. Code is available here https://github.com/VisMIL/CAMERAS