Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack

TL;DR

This paper introduces the minimum-margin (MM) attack, a fast and reliable method for evaluating adversarial robustness that significantly reduces computational costs while maintaining accuracy, facilitating broader practical use.

Contribution

The paper proposes the MM attack, a novel approach that evaluates adversarial robustness efficiently and reliably, with a new sequential target ranking method independent of class number.

Findings

MM attack achieves comparable robustness evaluation to AutoAttack

It reduces computational time to 3% of AutoAttack

Provides a practical tool for adversarial training applications

Abstract

The AutoAttack (AA) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available. However, the high computational cost (e.g., 100 times more than that of the project gradient descent attack) makes AA infeasible for practitioners with limited computational resources, and also hinders applications of AA in the adversarial training (AT). In this paper, we propose a novel method, minimum-margin (MM) attack, to fast and reliably evaluate adversarial robustness. Compared with AA, our method achieves comparable performance but only costs 3% of the computational time in extensive experiments. The reliability of our method lies in that we evaluate the quality of adversarial examples using the margin between two targets that can precisely identify the most adversarial example. The computational efficiency of our method lies in an effective Sequential TArget Ranking Selection (STARS) method, ensuring that the cost of the MM attack is independent of the number of classes. The MM attack opens a new way for evaluating adversarial robustness and provides a feasible and reliable way to generate high-quality adversarial examples in AT.