Boosting Fast Adversarial Training with Learnable Adversarial Initialization

TL;DR

This paper introduces a novel sample-dependent adversarial initialization for fast adversarial training, significantly improving robustness while maintaining efficiency, by jointly optimizing a generative network and the target model.

Contribution

It proposes a learnable, sample-dependent initialization method for fast adversarial training, enhancing robustness without increasing computational cost.

Findings

Outperforms state-of-the-art fast AT methods on four benchmarks.

Achieves robustness comparable to multi-step AT methods.

Demonstrates the effectiveness of learnable initialization in adversarial training.

Abstract

Adversarial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training. However, most AT methods are in face of expensive time and computational cost for calculating gradients at multiple steps in generating adversarial examples. To boost training efficiency, fast gradient sign method (FGSM) is adopted in fast AT methods by calculating gradient only once. Unfortunately, the robustness is far from satisfactory. One reason may arise from the initialization fashion. Existing fast AT generally uses a random sample-agnostic initialization, which facilitates the efficiency yet hinders a further robustness improvement. Up to now, the initialization in fast AT is still not extensively explored. In this paper, we boost fast AT with a sample-dependent adversarial initialization, i.e., an output from a generative network conditioned on a benign image and its gradient information from the target network. As the generative network and the target network are optimized jointly in the training phase, the former can adaptively generate an effective initialization with respect to the latter, which motivates gradually improved robustness. Experimental evaluations on four benchmark databases demonstrate the superiority of our proposed method over state-of-the-art fast AT methods, as well as comparable robustness to advanced multi-step AT methods. The code is released at https://github.com//jiaxiaojunQAQ//FGSM-SDI.