First-order Adversarial Vulnerability of Neural Networks and Input Dimension

TL;DR

This paper demonstrates that neural network vulnerability to adversarial attacks increases with input dimension, showing that larger images inherently pose greater risks regardless of network architecture or training methods.

Contribution

The study reveals that adversarial vulnerability is fundamentally linked to input dimension and is independent of network topology, providing theoretical and empirical insights into this phenomenon.

Findings

Vulnerability grows with the square root of input dimension at initialization.

Regularization can attenuate but not eliminate dimension-dependent vulnerability.

Vulnerability persists after standard and robust training.

Abstract

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: for many standard network architectures, we prove that at initialization, the $\ell_1$-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension dependence persists after either usual or robust training, but gets attenuated with higher regularization.