Interpreting Adversarial Examples by Activation Promotion and Suppression

TL;DR

This paper investigates how adversarial perturbations affect CNN activations, categorizing them into promotion, suppression, or balanced types, and links these effects to interpretability and robustness improvements.

Contribution

It introduces a promotion-suppression effect framework for understanding adversarial perturbations and connects pixel-level effects to class-specific regions and concept-level interpretability.

Findings

Adversarial perturbations can be categorized into suppression, promotion, or balanced types.

Pixel-level perturbations relate to class-specific discriminative regions via CAM.

Sensitivity of units to attacks correlates with their semantic interpretability.

Abstract

It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers. However, interpretability of these perturbations is less explored in the literature. This work aims to better understand the roles of adversarial perturbations and provide visual explanations from pixel, image and network perspectives. We show that adversaries have a promotion-suppression effect (PSE) on neurons' activations and can be primarily categorized into three types: i) suppression-dominated perturbations that mainly reduce the classification score of the true label, ii) promotion-dominated perturbations that focus on boosting the confidence of the target label, and iii) balanced perturbations that play a dual role in suppression and promotion. We also provide image-level interpretability of adversarial examples. This links PSE of pixel-level perturbations to class-specific discriminative image regions localized by class activation mapping (Zhou et al. 2016). Further, we examine the adversarial effect through network dissection (Bau et al. 2017), which offers concept-level interpretability of hidden units. We show that there exists a tight connection between the units' sensitivity to adversarial attacks and their interpretability on semantic concepts. Lastly, we provide some new insights from our interpretation to improve the adversarial robustness of networks.