Natural Adversarial Examples

TL;DR

This paper presents two challenging datasets, ImageNet-A and ImageNet-O, that reveal shared weaknesses in computer vision models and demonstrate the limited effectiveness of current robustness techniques.

Contribution

The authors introduce two new datasets created with adversarial filtration to test model robustness and out-of-distribution detection, highlighting shared weaknesses in existing models.

Findings

ImageNet-A significantly reduces model accuracy to around 2%.

Models perform near chance on out-of-distribution detection with ImageNet-O.

Current data augmentation techniques offer limited robustness improvements.

Abstract

We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtration technique to create datasets with limited spurious cues. Our datasets' real-world, unmodified examples transfer to various unseen models reliably, demonstrating that computer vision models have shared weaknesses. The first dataset is called ImageNet-A and is like the ImageNet test set, but it is far more challenging for existing models. We also curate an adversarial out-of-distribution detection dataset called ImageNet-O, which is the first out-of-distribution detection dataset created for ImageNet models. On ImageNet-A a DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90%, and its out-of-distribution detection performance on ImageNet-O is near random chance levels. We find that existing data augmentation techniques hardly boost performance, and using other public training datasets provides improvements that are limited. However, we find that improvements to computer vision architectures provide a promising path towards robust models.