Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness

TL;DR

This paper uncovers a subtle cause of overestimated adversarial robustness called imbalanced gradients, introduces a new attack method to exploit this issue, and demonstrates its effectiveness on multiple defense models.

Contribution

It identifies imbalanced gradients as a hidden factor in robustness overestimation and proposes the Margin Decomposition attack to address this problem.

Findings

11 out of 24 models are affected by imbalanced gradients

MD attack reduces robustness estimates by over 1% on average

Provides insights and countermeasures for imbalanced gradients

Abstract

Evaluating the robustness of a defense model is a challenging task in adversarial robustness research. Obfuscated gradients have previously been found to exist in many defense methods and cause a false signal of robustness. In this paper, we identify a more subtle situation called Imbalanced Gradients that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a Margin Decomposition (MD) attack that decomposes a margin loss into individual terms and then explores the attackability of these terms separately via a two-stage process. We also propose a multi-targeted and ensemble version of our MD attack. By investigating 24 defense models proposed since 2018, we find that 11 models are susceptible to a certain degree of imbalanced gradients and our MD attack can decrease their robustness evaluated by the best standalone baseline attack by more than 1%. We also provide an in-depth investigation on the likely causes of imbalanced gradients and effective countermeasures. Our code is available at https://github.com/HanxunH/MDAttack.