Intriguing Properties of Robust Classification

TL;DR

This paper investigates the fundamental limitations of training robust classifiers, revealing that achieving robustness often requires exponentially more data than achieving accuracy, and that data quantity is crucial for robust generalization.

Contribution

The paper presents a theoretical analysis showing the necessity of enormous data for robust classification and empirically demonstrates the impact of data size on robustness in vision datasets.

Findings

Robust classifiers may require exponentially more data than accurate classifiers.

Data quantity is the key factor influencing robust performance.

Certain data directions useful for non-robust generalization are inaccessible to robust classifiers.

Abstract

Despite extensive research since the community learned about adversarial examples 10 years ago, we still do not know how to train high-accuracy classifiers that are guaranteed to be robust to small perturbations of their inputs. Previous works often argued that this might be because no classifier exists that is robust and accurate at the same time. However, in computer vision this assumption does not match reality where humans are usually accurate and robust on most tasks of interest. We offer an alternative explanation and show that in certain settings robust generalization is only possible with unrealistically large amounts of data. Specifically, we find a setting where a robust classifier exists, it is easy to learn an accurate classifier, yet it requires an exponential amount of data to learn a robust classifier. Based on this theoretical result, we evaluate the influence of the amount of training data on datasets such as CIFAR-10. Our findings indicate that the amount of training data is the main factor determining the robust performance. Furthermore we show that there are low magnitude directions in the data which are useful for non-robust generalization but are not available for robust classifiers. We provide code at https://github.com/berndprach/IntriguingProperties.