Computational Asymmetries in Robust Classification

TL;DR

This paper explores the computational complexity of adversarial robustness in classifiers, revealing asymmetries between attack and defense hardness, and introduces methods and datasets for robustness certification and evaluation.

Contribution

It proves complexity asymmetries in adversarial robustness, introduces Counter-Attack for inference-time certification, and releases UG100 dataset for attack benchmarking.

Findings

Robust training is NP-hard, attack is Sigma2_P-hard, explaining frequent fooling.

Counter-Attack reverses asymmetry, defense is NP-hard, attack remains Sigma2_P-hard.

Adversarial attacks can be used effectively for robustness certification.

Abstract

In the context of adversarial robustness, we make three strongly related contributions. First, we prove that while attacking ReLU classifiers is $\mathit{NP}$-hard, ensuring their robustness at training time is $\Sigma^2_P$-hard (even on a single example). This asymmetry provides a rationale for the fact that robust classifications approaches are frequently fooled in the literature. Second, we show that inference-time robustness certificates are not affected by this asymmetry, by introducing a proof-of-concept approach named Counter-Attack (CA). Indeed, CA displays a reversed asymmetry: running the defense is $\mathit{NP}$-hard, while attacking it is $\Sigma_2^P$-hard. Finally, motivated by our previous result, we argue that adversarial attacks can be used in the context of robustness certification, and provide an empirical evaluation of their effectiveness. As a byproduct of this process, we also release UG100, a benchmark dataset for adversarial attacks.