Generalizing Adversarial Explanations with Grad-CAM

TL;DR

This paper extends Grad-CAM from example-based explanations to a global model behavior explanation using new metrics, enabling better understanding of CNNs under adversarial attacks.

Contribution

Introduces two novel metrics, MOD and VID, to generalize Grad-CAM for explaining overall CNN behavior and robustness against adversarial attacks.

Findings

Grad-CAM heatmaps shift under adversarial attacks across multiple CNN models.

The proposed metrics effectively capture changes in model decision regions.

Method aids in understanding and explaining black box CNN models in adversarial contexts.

Abstract

Gradient-weighted Class Activation Mapping (Grad- CAM), is an example-based explanation method that provides a gradient activation heat map as an explanation for Convolution Neural Network (CNN) models. The drawback of this method is that it cannot be used to generalize CNN behaviour. In this paper, we present a novel method that extends Grad-CAM from example-based explanations to a method for explaining global model behaviour. This is achieved by introducing two new metrics, (i) Mean Observed Dissimilarity (MOD) and (ii) Variation in Dissimilarity (VID), for model generalization. These metrics are computed by comparing a Normalized Inverted Structural Similarity Index (NISSIM) metric of the Grad-CAM generated heatmap for samples from the original test set and samples from the adversarial test set. For our experiment, we study adversarial attacks on deep models such as VGG16, ResNet50, and ResNet101, and wide models such as InceptionNetv3 and XceptionNet using Fast Gradient Sign Method (FGSM). We then compute the metrics MOD and VID for the automatic face recognition (AFR) use case with the VGGFace2 dataset. We observe a consistent shift in the region highlighted in the Grad-CAM heatmap, reflecting its participation to the decision making, across all models under adversarial attacks. The proposed method can be used to understand adversarial attacks and explain the behaviour of black box CNN models for image analysis.