Rethinking gradient weights' influence over saliency map estimation

TL;DR

This paper introduces a novel method for saliency map estimation that uses a global guidance map to improve interpretability and specificity of gradient-based visual explanations in deep neural networks.

Contribution

It proposes a new rectification technique for gradient-based saliency maps using a global guidance map, enhancing interpretability and specificity over traditional methods.

Findings

Significant improvement in saliency map quality on ImageNet, MS-COCO 14, and PASCAL VOC 2012 datasets.

Outperforms eight existing saliency visualizers in quantitative evaluations.

Produces cleaner and more instance-specific explanations.

Abstract

Class activation map (CAM) helps to formulate saliency maps that aid in interpreting the deep neural network's prediction. Gradient-based methods are generally faster than other branches of vision interpretability and independent of human guidance. The performance of CAM-like studies depends on the governing model's layer response, and the influences of the gradients. Typical gradient-oriented CAM studies rely on weighted aggregation for saliency map estimation by projecting the gradient maps into single weight values, which may lead to over generalized saliency map. To address this issue, we use a global guidance map to rectify the weighted aggregation operation during saliency estimation, where resultant interpretations are comparatively clean er and instance-specific. We obtain the global guidance map by performing elementwise multiplication between the feature maps and their corresponding gradient maps. To validate our study, we compare the proposed study with eight different saliency visualizers. In addition, we use seven commonly used evaluation metrics for quantitative comparison. The proposed scheme achieves significant improvement over the test images from the ImageNet, MS-COCO 14, and PASCAL VOC 2012 datasets.