On the Structural Sensitivity of Deep Convolutional Networks to the Directions of Fourier Basis Functions

TL;DR

This paper reveals that deep convolutional networks are highly sensitive to Fourier basis function directions, impacting their robustness and generalization, and introduces an algorithm for shift-invariant adversarial perturbations.

Contribution

It demonstrates the sensitivity of convolutional networks to Fourier basis directions and derives this property through theoretical analysis and empirical validation.

Findings

Convolutional networks are sensitive to Fourier basis function directions.

The paper proposes an algorithm for shift-invariant universal adversarial perturbations.

Sensitivity to Fourier directions affects network robustness and generalization.

Abstract

Data-agnostic quasi-imperceptible perturbations on inputs are known to degrade recognition accuracy of deep convolutional networks severely. This phenomenon is considered to be a potential security issue. Moreover, some results on statistical generalization guarantees indicate that the phenomenon can be a key to improve the networks' generalization. However, the characteristics of the shared directions of such harmful perturbations remain unknown. Our primal finding is that convolutional networks are sensitive to the directions of Fourier basis functions. We derived the property by specializing a hypothesis of the cause of the sensitivity, known as the linearity of neural networks, to convolutional networks and empirically validated it. As a by-product of the analysis, we propose an algorithm to create shift-invariant universal adversarial perturbations available in black-box settings.