Adversarial Examples Are a Natural Consequence of Test Error in Noise

TL;DR

This paper demonstrates that adversarial examples and image corruption robustness are fundamentally linked phenomena, suggesting that improving defenses against adversarial attacks should also enhance performance on naturally corrupted images.

Contribution

It provides empirical and theoretical evidence connecting adversarial robustness with corruption robustness, advocating for joint improvements and evaluation strategies.

Findings

Adversarial examples and image corruptions share underlying causes.

Robustness to adversarial attacks correlates with robustness to natural corruptions.

Recommendations for evaluating defenses using benchmarks like Imagenet-C.

Abstract

Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to small modifications of a correctly handled input. Less surprisingly, image classifiers also lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise. In this paper we provide both empirical and theoretical evidence that these are two manifestations of the same underlying phenomenon, establishing close connections between the adversarial robustness and corruption robustness research programs. This suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions. Based on our results we recommend that future adversarial defenses consider evaluating the robustness of their methods to distributional shift with benchmarks such as Imagenet-C.