Natural Adversarial Objects

TL;DR

This paper introduces the Natural Adversarial Objects dataset to evaluate object detection model robustness, revealing that high accuracy on standard benchmarks does not ensure robustness against real-world adversarial-like examples.

Contribution

The paper presents a new dataset, NAO, for assessing object detection robustness and analyzes why models fail on these real-world challenging examples.

Findings

Models' mAP drops significantly on NAO compared to MSCOCO.

Better MSCOCO performance does not guarantee robustness on NAO.

Models are overly sensitive to local textures and rely mainly on pixel information within bounding boxes.

Abstract

Although state-of-the-art object detection methods have shown compelling performance, models often are not robust to adversarial attacks and out-of-distribution data. We introduce a new dataset, Natural Adversarial Objects (NAO), to evaluate the robustness of object detection models. NAO contains 7,934 images and 9,943 objects that are unmodified and representative of real-world scenarios, but cause state-of-the-art detection models to misclassify with high confidence. The mean average precision (mAP) of EfficientDet-D7 drops 74.5% when evaluated on NAO compared to the standard MSCOCO validation set. Moreover, by comparing a variety of object detection architectures, we find that better performance on MSCOCO validation set does not necessarily translate to better performance on NAO, suggesting that robustness cannot be simply achieved by training a more accurate model. We further investigate why examples in NAO are difficult to detect and classify. Experiments of shuffling image patches reveal that models are overly sensitive to local texture. Additionally, using integrated gradients and background replacement, we find that the detection model is reliant on pixel information within the bounding box, and insensitive to the background context when predicting class labels. NAO can be downloaded at https://drive.google.com/drive/folders/15P8sOWoJku6SSEiHLEts86ORfytGezi8.