Are adversarial examples inevitable?

TL;DR

This paper investigates whether adversarial examples are unavoidable in neural networks by providing theoretical bounds on their susceptibility and exploring how problem complexity affects robustness.

Contribution

It offers a theoretical analysis demonstrating the inevitability of adversarial examples for certain problem classes and examines practical implications for real-world classifiers.

Findings

Adversarial examples are inescapable for some problem classes.

Dimensionality and image complexity limit classifier robustness.

Theoretical bounds inform the fundamental limits of adversarial defenses.

Abstract

A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.