Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain

TL;DR

This paper investigates the role of phase spectrum in CNN robustness, proposing a novel data augmentation method that recombines phase and amplitude spectra to improve generalization and resistance to perturbations.

Contribution

It introduces a new perspective on CNN robustness by emphasizing phase spectrum importance and proposes a phase-amplitude recombination technique for data augmentation.

Findings

Achieves state-of-the-art results on robustness and calibration tasks.

Improves out-of-distribution detection and adversarial robustness.

Enhances adaptability to common corruptions and surface variations.

Abstract

Recently, the generalization behavior of Convolutional Neural Networks (CNN) is gradually transparent through explanation techniques with the frequency components decomposition. However, the importance of the phase spectrum of the image for a robust vision system is still ignored. In this paper, we notice that the CNN tends to converge at the local optimum which is closely related to the high-frequency components of the training images, while the amplitude spectrum is easily disturbed such as noises or common corruptions. In contrast, more empirical studies found that humans rely on more phase components to achieve robust recognition. This observation leads to more explanations of the CNN's generalization behaviors in both robustness to common perturbations and out-of-distribution detection, and motivates a new perspective on data augmentation designed by re-combing the phase spectrum of the current image and the amplitude spectrum of the distracter image. That is, the generated samples force the CNN to pay more attention to the structured information from phase components and keep robust to the variation of the amplitude. Experiments on several image datasets indicate that the proposed method achieves state-of-the-art performances on multiple generalizations and calibration tasks, including adaptability for common corruptions and surface variations, out-of-distribution detection, and adversarial attack.