Most Convolutional Networks Suffer from Small Adversarial Perturbations

TL;DR

This paper proves that small adversarial perturbations in convolutional neural networks can be found close to the input, using Fourier analysis to bound the singular values of convolutional operators, extending prior work on CNN adversarial examples.

Contribution

It extends previous results by showing that adversarial examples in random CNNs can be found at near-minimal distances using a single gradient step, with bounds derived via Fourier analysis.

Findings

Adversarial examples exist within -distance of  (x) in random CNNs.

Small perturbations can be found with a single gradient step.

Fourier decomposition bounds the singular values of convolutional operators.

Abstract

The existence of adversarial examples is relatively understood for random fully connected neural networks, but much less so for convolutional neural networks (CNNs). The recent work [Daniely, 2025] establishes that adversarial examples can be found in CNNs, in some non-optimal distance from the input. We extend over this work and prove that adversarial examples in random CNNs with input dimension $d$ can be found already in $\ell_2$-distance of order $\lVert x \rVert /\sqrt{d}$ from the input $x$, which is essentially the nearest possible. We also show that such adversarial small perturbations can be found using a single step of gradient descent. To derive our results we use Fourier decomposition to efficiently bound the singular values of a random linear convolutional operator, which is the main ingredient of a CNN layer. This bound might be of independent interest.