An Alternative Surrogate Loss for PGD-based Adversarial Testing

TL;DR

This paper introduces MultiTargeted, an alternative surrogate loss for PGD-based adversarial testing, which outperforms existing methods and ranks first on multiple adversarial robustness leaderboards.

Contribution

It proposes MultiTargeted, a novel surrogate loss for PGD adversarial testing, with guarantees of optimality and improved efficiency over prior methods.

Findings

MultiTargeted outperforms existing PGD variants in adversarial testing.

It ranks first on MadryLab's MNIST and CIFAR-10 leaderboards.

MultiTargeted achieves lower model accuracy under adversarial attacks.

Abstract

Adversarial testing methods based on Projected Gradient Descent (PGD) are widely used for searching norm-bounded perturbations that cause the inputs of neural networks to be misclassified. This paper takes a deeper look at these methods and explains the effect of different hyperparameters (i.e., optimizer, step size and surrogate loss). We introduce the concept of MultiTargeted testing, which makes clever use of alternative surrogate losses, and explain when and how MultiTargeted is guaranteed to find optimal perturbations. Finally, we demonstrate that MultiTargeted outperforms more sophisticated methods and often requires less iterative steps than other variants of PGD found in the literature. Notably, MultiTargeted ranks first on MadryLab's white-box MNIST and CIFAR-10 leaderboards, reducing the accuracy of their MNIST model to 88.36% (with $\ell_\infty$ perturbations of $\epsilon = 0.3$) and the accuracy of their CIFAR-10 model to 44.03% (at $\epsilon = 8/255$). MultiTargeted also ranks first on the TRADES leaderboard reducing the accuracy of their CIFAR-10 model to 53.07% (with $\ell_\infty$ perturbations of $\epsilon = 0.031$).