Gaussian Smoothing in Saliency Maps: The Stability-Fidelity Trade-Off in Neural Network Interpretability

TL;DR

This paper investigates how Gaussian smoothing affects the stability and fidelity of saliency maps in neural network interpretability, revealing a trade-off between robustness to training randomness and faithfulness to the original explanation.

Contribution

It extends the stability analysis framework to gradient-based saliency maps and quantifies the stability-fidelity trade-off introduced by Gaussian smoothing.

Findings

Gaussian smoothing increases stability of saliency maps.

Higher smoothing reduces the faithfulness of explanations.

Empirical results confirm the stability-fidelity trade-off.

Abstract

Saliency maps have been widely used to interpret the decisions of neural network classifiers and discover phenomena from their learned functions. However, standard gradient-based maps are frequently observed to be highly sensitive to the randomness of training data and the stochasticity in the training process. In this work, we study the role of Gaussian smoothing in the well-known Smooth-Grad algorithm in the stability of the gradient-based maps to the randomness of training samples. We extend the algorithmic stability framework to gradient-based interpretation maps and prove bounds on the stability error of standard Simple-Grad, Integrated-Gradients, and Smooth-Grad saliency maps. Our theoretical results suggest the role of Gaussian smoothing in boosting the stability of gradient-based maps to the randomness of training settings. On the other hand, we analyze the faithfulness of the Smooth-Grad maps to the original Simple-Grad and show the lower fidelity under a more intense Gaussian smoothing. We support our theoretical results by performing several numerical experiments on standard image datasets. Our empirical results confirm our hypothesis on the fidelity-stability trade-off in the application of Gaussian smoothing to gradient-based interpretation maps.