Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy

TL;DR

This paper empirically investigates class-wise accuracy and robustness in adversarially trained CNNs, revealing inter-class discrepancies that are consistent across datasets and models, and exploring potential mitigation strategies.

Contribution

It provides the first comprehensive analysis of class-wise robustness discrepancies in adversarial training, highlighting their universality and potential causes.

Findings

Inter-class accuracy and robustness vary significantly even with balanced datasets.

Adversarial training tends to increase inter-class robustness discrepancies.

Discrepancies are consistent across different datasets, models, and hyper-parameters.

Abstract

Convolutional neural networks (CNNs) have made significant advancement, however, they are widely known to be vulnerable to adversarial attacks. Adversarial training is the most widely used technique for improving adversarial robustness to strong white-box attacks. Prior works have been evaluating and improving the model average robustness without class-wise evaluation. The average evaluation alone might provide a false sense of robustness. For example, the attacker can focus on attacking the vulnerable class, which can be dangerous, especially, when the vulnerable class is a critical one, such as "human" in autonomous driving. We propose an empirical study on the class-wise accuracy and robustness of adversarially trained models. We find that there exists inter-class discrepancy for accuracy and robustness even when the training dataset has an equal number of samples for each class. For example, in CIFAR10, "cat" is much more vulnerable than other classes. Moreover, this inter-class discrepancy also exists for normally trained models, while adversarial training tends to further increase the discrepancy. Our work aims to investigate the following questions: (a) is the phenomenon of inter-class discrepancy universal regardless of datasets, model architectures and optimization hyper-parameters? (b) If so, what can be possible explanations for the inter-class discrepancy? (c) Can the techniques proposed in the long tail classification be readily extended to adversarial training for addressing the inter-class discrepancy?