Wavelet Regularization Benefits Adversarial Training

TL;DR

This paper introduces a wavelet regularization technique using Haar wavelets to improve adversarial robustness of neural networks by focusing on high-frequency vulnerability regulation in the frequency domain.

Contribution

It proposes a novel wavelet regularization method integrated into neural networks, enhancing adversarial training effectiveness especially in deep wide models.

Findings

Improved robustness on CIFAR-10 and CIFAR-100 datasets.

Wavelet regularization enhances adversarial defense in deep wide networks.

Visualization confirms the effectiveness of frequency domain regulation.

Abstract

Adversarial training methods are state-of-the-art (SOTA) empirical defense methods against adversarial examples. Many regularization methods have been proven to be effective with the combination of adversarial training. Nevertheless, such regularization methods are implemented in the time domain. Since adversarial vulnerability can be regarded as a high-frequency phenomenon, it is essential to regulate the adversarially-trained neural network models in the frequency domain. Faced with these challenges, we make a theoretical analysis on the regularization property of wavelets which can enhance adversarial training. We propose a wavelet regularization method based on the Haar wavelet decomposition which is named Wavelet Average Pooling. This wavelet regularization module is integrated into the wide residual neural network so that a new WideWaveletResNet model is formed. On the datasets of CIFAR-10 and CIFAR-100, our proposed Adversarial Wavelet Training method realizes considerable robustness under different types of attacks. It verifies the assumption that our wavelet regularization method can enhance adversarial robustness especially in the deep wide neural networks. The visualization experiments of the Frequency Principle (F-Principle) and interpretability are implemented to show the effectiveness of our method. A detailed comparison based on different wavelet base functions is presented. The code is available at the repository: \url{https://github.com/momo1986/AdversarialWaveletTraining}.