On the Effects of Adversarial Perturbations on Distribution Robustness

TL;DR

This paper provides a theoretical analysis of the tradeoff between adversarial and distribution robustness, revealing conditions under which distribution robustness can improve despite adversarial training, emphasizing the role of feature separability.

Contribution

It introduces a tractable surrogate for adversarial training analysis and uncovers nuanced effects of perturbations on distribution robustness, especially related to feature separability.

Findings

Adversarial training can harm distribution robustness but may improve it under certain data biases.

Moderate bias in data can lead to increased distribution robustness with $\, ext{l}_ ext{infty}$ perturbations.

Greater feature separability enhances distribution robustness even in skewed data scenarios.

Abstract

Adversarial robustness refers to a model's ability to resist perturbation of inputs, while distribution robustness evaluates the performance of the model under data shifts. Although both aim to ensure reliable performance, prior work has revealed a tradeoff in distribution and adversarial robustness. Specifically, adversarial training might increase reliance on spurious features, which can harm distribution robustness, especially the performance on some underrepresented subgroups. We present a theoretical analysis of adversarial and distribution robustness that provides a tractable surrogate for per-step adversarial training by studying models trained on perturbed data. In addition to the tradeoff, our work further identified a nuanced phenomenon that $\ell_\infty$ perturbations on data with moderate bias can yield an increase in distribution robustness. Moreover, the gain in distribution robustness remains on highly skewed data when simplicity bias induces reliance on the core feature, characterized as greater feature separability. Our theoretical analysis extends the understanding of the tradeoff by highlighting the interplay of the tradeoff and the feature separability. Despite the tradeoff that persists in many cases, overlooking the role of feature separability may lead to misleading conclusions about robustness.