Scale-Invariant Adversarial Attack for Evaluating and Enhancing Adversarial Defenses

TL;DR

This paper introduces SI-PGD, a scale-invariant adversarial attack based on angular features, which effectively evaluates and enhances defenses by remaining stable under logit rescaling, leading to improved robustness and defense performance.

Contribution

The paper proposes SI-PGD, a novel scale-invariant attack using angular features, and a defense mechanism based on cosine angles, improving evaluation and robustness of models against adversarial attacks.

Findings

SI-PGD outperforms existing attacks in effectiveness.

The SI defense mechanism achieves state-of-the-art robustness.

The approach remains stable under logit rescaling.

Abstract

Efficient and effective attacks are crucial for reliable evaluation of defenses, and also for developing robust models. Projected Gradient Descent (PGD) attack has been demonstrated to be one of the most successful adversarial attacks. However, the effect of the standard PGD attack can be easily weakened by rescaling the logits, while the original decision of every input will not be changed. To mitigate this issue, in this paper, we propose Scale-Invariant Adversarial Attack (SI-PGD), which utilizes the angle between the features in the penultimate layer and the weights in the softmax layer to guide the generation of adversaries. The cosine angle matrix is used to learn angularly discriminative representation and will not be changed with the rescaling of logits, thus making SI-PGD attack to be stable and effective. We evaluate our attack against multiple defenses and show improved performance when compared with existing attacks. Further, we propose Scale-Invariant (SI) adversarial defense mechanism based on the cosine angle matrix, which can be embedded into the popular adversarial defenses. The experimental results show the defense method with our SI mechanism achieves state-of-the-art performance among multi-step and single-step defenses.