Verifying the Causes of Adversarial Examples

TL;DR

This paper investigates the underlying causes of adversarial examples in neural networks through controlled experiments, identifying geometry and statistical factors as key contributors to their formation.

Contribution

It systematically verifies potential causes of adversarial examples and introduces new techniques to control these effects, enhancing understanding of their origins.

Findings

Geometric factors are primary causes of adversarial examples.

Statistical factors amplify adversarial phenomena, especially at high confidence levels.

Controlled experiments validate the influence of model linearity and category geometry.

Abstract

The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs, which mislead a classifier to incorrect outputs in high confidence. Limited by the extreme difficulty in examining a high-dimensional image space thoroughly, research on explaining and justifying the causes of adversarial examples falls behind studies on attacks and defenses. In this paper, we present a collection of potential causes of adversarial examples and verify (or partially verify) them through carefully-designed controlled experiments. The major causes of adversarial examples include model linearity, one-sum constraint, and geometry of the categories. To control the effect of those causes, multiple techniques are applied such as $L_2$ normalization, replacement of loss functions, construction of reference datasets, and novel models using multi-layer perceptron probabilistic neural networks (MLP-PNN) and density estimation (DE). Our experiment results show that geometric factors tend to be more direct causes and statistical factors magnify the phenomenon, especially for assigning high prediction confidence. We believe this paper will inspire more studies to rigorously investigate the root causes of adversarial examples, which in turn provide useful guidance on designing more robust models.