A3T: Accuracy Aware Adversarial Training

TL;DR

This paper investigates overfitting in adversarial training, identifies its causes, and proposes a new method that improves generalization across multiple domains while maintaining robustness.

Contribution

It introduces a novel adversarial training approach that mitigates overfitting by leveraging misclassified samples, enhancing generalization without sacrificing robustness.

Findings

Improved generalization in adversarial training across vision, NLP, and tabular tasks.

Comparable robustness to state-of-the-art methods.

Better mitigation of overfitting compared to traditional approaches.

Abstract

Adversarial training has been empirically shown to be more prone to overfitting than standard training. The exact underlying reasons still need to be fully understood. In this paper, we identify one cause of overfitting related to current practices of generating adversarial samples from misclassified samples. To address this, we propose an alternative approach that leverages the misclassified samples to mitigate the overfitting problem. We show that our approach achieves better generalization while having comparable robustness to state-of-the-art adversarial training methods on a wide range of computer vision, natural language processing, and tabular tasks.