Regularizers for Single-step Adversarial Training

TL;DR

This paper introduces three regularizers that improve single-step adversarial training, enabling models to achieve robustness comparable to multi-step methods while avoiding gradient masking issues.

Contribution

The paper proposes novel regularizers that enhance single-step adversarial training, addressing gradient masking and robustness limitations.

Findings

Regularizers effectively mitigate gradient masking effects.

Models trained with regularizers match multi-step adversarial training robustness.

Proposed methods are computationally efficient and scalable.

Abstract

The progress in the last decade has enabled machine learning models to achieve impressive performance across a wide range of tasks in Computer Vision. However, a plethora of works have demonstrated the susceptibility of these models to adversarial samples. Adversarial training procedure has been proposed to defend against such adversarial attacks. Adversarial training methods augment mini-batches with adversarial samples, and typically single-step (non-iterative) methods are used for generating these adversarial samples. However, models trained using single-step adversarial training converge to degenerative minima where the model merely appears to be robust. The pseudo robustness of these models is due to the gradient masking effect. Although multi-step adversarial training helps to learn robust models, they are hard to scale due to the use of iterative methods for generating adversarial samples. To address these issues, we propose three different types of regularizers that help to learn robust models using single-step adversarial training methods. The proposed regularizers mitigate the effect of gradient masking by harnessing on properties that differentiate a robust model from that of a pseudo robust model. Performance of models trained using the proposed regularizers is on par with models trained using computationally expensive multi-step adversarial training methods.