THAT: Two Head Adversarial Training for Improving Robustness at Scale

TL;DR

THAT introduces a two-head adversarial training framework that enhances robustness and accuracy on large-scale datasets like ImageNet, outperforming existing methods in adversarial settings.

Contribution

The paper presents a novel two-head adversarial training approach tailored for large-scale, many-class datasets, improving robustness without sacrificing natural accuracy.

Findings

Achieves state-of-the-art robust accuracy on ImageNet.

Outperforms alternative adversarial training methods.

Maintains high natural accuracy under adversarial conditions.

Abstract

Many variants of adversarial training have been proposed, with most research focusing on problems with relatively few classes. In this paper, we propose Two Head Adversarial Training (THAT), a two-stream adversarial learning network that is designed to handle the large-scale many-class ImageNet dataset. The proposed method trains a network with two heads and two loss functions; one to minimize feature-space domain shift between natural and adversarial images, and one to promote high classification accuracy. This combination delivers a hardened network that achieves state of the art robust accuracy while maintaining high natural accuracy on ImageNet. Through extensive experiments, we demonstrate that the proposed framework outperforms alternative methods under both standard and "free" adversarial training settings.