Enhancing Deep Neural Network Saliency Visualizations with Gradual Extrapolation

TL;DR

This paper introduces Gradual Extrapolation, a technique that refines saliency maps in neural networks by sharpening heatmaps, leading to more accurate and interpretable visual explanations with minimal additional computational cost.

Contribution

It presents a novel method to enhance class activation maps by gradually propagating and sharpening the heatmaps, improving localization accuracy across various neural network models.

Findings

Significantly improves localization detection accuracy.

Maintains low additional computational costs.

Applicable to multiple deep neural network architectures.

Abstract

In this paper, an enhancement technique for the class activation mapping methods such as gradient-weighted class activation maps or excitation backpropagation is proposed to present the visual explanations of decisions from convolutional neural network-based models. The proposed idea, called Gradual Extrapolation, can supplement any method that generates a heatmap picture by sharpening the output. Instead of producing a coarse localization map that highlights the important predictive regions in the image, the proposed method outputs the specific shape that most contributes to the model output. Thus, the proposed method improves the accuracy of saliency maps. The effect has been achieved by the gradual propagation of the crude map obtained in the deep layer through all preceding layers with respect to their activations. In validation tests conducted on a selected set of images, the faithfulness, interpretability, and applicability of the method are evaluated. The proposed technique significantly improves the localization detection of the neural networks attention at low additional computational costs. Furthermore, the proposed method is applicable to a variety deep neural network models. The code for the method can be found at https://github.com/szandala/gradual-extrapolation