Hard Adversarial Example Mining for Improving Robust Fairness

TL;DR

This paper introduces HAM, a framework that improves the fairness of adversarial training by focusing on hard adversarial examples, reducing overconfidence issues, and enhancing robustness across multiple datasets.

Contribution

HAM is a novel adaptive hard adversarial example mining method that enhances robustness and fairness in adversarial training by selectively focusing on challenging examples.

Findings

HAM improves robust fairness significantly.

HAM reduces computational cost of adversarial training.

Experimental results outperform existing methods on CIFAR-10, SVHN, and Imagenette.

Abstract

Adversarial training (AT) is widely considered the state-of-the-art technique for improving the robustness of deep neural networks (DNNs) against adversarial examples (AE). Nevertheless, recent studies have revealed that adversarially trained models are prone to unfairness problems, restricting their applicability. In this paper, we empirically observe that this limitation may be attributed to serious adversarial confidence overfitting, i.e., certain adversarial examples with overconfidence. To alleviate this problem, we propose HAM, a straightforward yet effective framework via adaptive Hard Adversarial example Mining.HAM concentrates on mining hard adversarial examples while discarding the easy ones in an adaptive fashion. Specifically, HAM identifies hard AEs in terms of their step sizes needed to cross the decision boundary when calculating loss value. Besides, an early-dropping mechanism is incorporated to discard the easy examples at the initial stages of AE generation, resulting in efficient AT. Extensive experimental results on CIFAR-10, SVHN, and Imagenette demonstrate that HAM achieves significant improvement in robust fairness while reducing computational cost compared to several state-of-the-art adversarial training methods. The code will be made publicly available.