Testing predictions of representation cost theory with CNNs

TL;DR

This paper combines theoretical analysis and experiments to explain CNNs' sensitivity to low-frequency signals as a consequence of natural image frequency distributions, advancing understanding of model robustness.

Contribution

It introduces a novel theoretical framework analyzing CNN layer representations in frequency space to explain frequency sensitivity related to natural image statistics.

Findings

CNN sensitivity to low frequencies is due to natural image frequency distribution

Theoretical analysis in frequency space explains robustness characteristics

Experimental results support the frequency bias explanation

Abstract

It is widely acknowledged that trained convolutional neural networks (CNNs) have different levels of sensitivity to signals of different frequency. In particular, a number of empirical studies have documented CNNs sensitivity to low-frequency signals. In this work we show with theory and experiments that this observed sensitivity is a consequence of the frequency distribution of natural images, which is known to have most of its power concentrated in low-to-mid frequencies. Our theoretical analysis relies on representations of the layers of a CNN in frequency space, an idea that has previously been used to accelerate computations and study implicit bias of network training algorithms, but to the best of our knowledge has not been applied in the domain of model robustness.