Adversarial Robustness through Local Linearization

TL;DR

This paper introduces a new regularizer that promotes local linearity in neural networks, improving adversarial robustness and training efficiency, and achieving state-of-the-art results on CIFAR-10 and ImageNet datasets.

Contribution

The authors propose a novel regularizer that reduces gradient obfuscation, enabling faster training and enhanced robustness against adversarial attacks.

Findings

Achieves 47% adversarial accuracy on ImageNet with strong attacks.

Models trained with the regularizer avoid gradient obfuscation.

Faster training compared to traditional adversarial training.

Abstract

Adversarial training is an effective methodology for training deep neural networks that are robust against adversarial, norm-bounded perturbations. However, the computational cost of adversarial training grows prohibitively as the size of the model and number of input dimensions increase. Further, training against less expensive and therefore weaker adversaries produces models that are robust against weak attacks but break down under attacks that are stronger. This is often attributed to the phenomenon of gradient obfuscation; such models have a highly non-linear loss surface in the vicinity of training examples, making it hard for gradient-based attacks to succeed even though adversarial examples still exist. In this work, we introduce a novel regularizer that encourages the loss to behave linearly in the vicinity of the training data, thereby penalizing gradient obfuscation while encouraging robustness. We show via extensive experiments on CIFAR-10 and ImageNet, that models trained with our regularizer avoid gradient obfuscation and can be trained significantly faster than adversarial training. Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with l-infinity adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack. Additionally, we match state of the art results for CIFAR-10 at 8/255.