Measuring the Transferability of $\ell_\infty$ Attacks by the $\ell_2$ Norm

TL;DR

This paper argues that evaluating adversarial attack strength solely by the $\, ext{l}_ ext{infty}\,$ norm is insufficient, and proposes measuring attack transferability using both $\, ext{l}_ ext{infty}\,$ and $\, ext{l}_ ext{2}\,$ norms, supported by extensive experiments.

Contribution

It introduces a combined measurement approach using both $\, ext{l}_ ext{infty}\,$ and $\, ext{l}_2\,$ norms to better assess adversarial attack transferability.

Findings

Larger $\, ext{l}_2\,$ distances improve transferability.

Existing methods craft perturbations with 70-130% larger $\, ext{l}_2\,$ distances.

Combined norm measurement provides deeper understanding of attack mechanisms.

Abstract

Deep neural networks could be fooled by adversarial examples with trivial differences to original samples. To keep the difference imperceptible in human eyes, researchers bound the adversarial perturbations by the $\ell_\infty$ norm, which is now commonly served as the standard to align the strength of different attacks for a fair comparison. However, we propose that using the $\ell_\infty$ norm alone is not sufficient in measuring the attack strength, because even with a fixed $\ell_\infty$ distance, the $\ell_2$ distance also greatly affects the attack transferability between models. Through the discovery, we reach more in-depth understandings towards the attack mechanism, i.e., several existing methods attack black-box models better partly because they craft perturbations with 70% to 130% larger $\ell_2$ distances. Since larger perturbations naturally lead to better transferability, we thereby advocate that the strength of attacks should be simultaneously measured by both the $\ell_\infty$ and $\ell_2$ norm. Our proposal is firmly supported by extensive experiments on ImageNet dataset from 7 attacks, 4 white-box models, and 9 black-box models.