Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods

TL;DR

This paper demonstrates that existing adversarial example detection methods can be bypassed, revealing that adversarial examples are more difficult to detect than previously believed and challenging assumptions about their intrinsic properties.

Contribution

It provides a comprehensive survey of ten detection methods, shows they can all be defeated with new loss functions, and offers guidelines for evaluating future defenses.

Findings

All ten detection methods can be bypassed.

Adversarial examples are harder to detect than previously thought.

Properties of adversarial examples are not intrinsic.

Abstract

Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.