Scaling Adversarial Training via Data Selection

TL;DR

This paper introduces Selective Adversarial Training, which reduces computational costs by focusing on critical samples during adversarial training, maintaining robustness while halving the computation.

Contribution

It proposes two principled sample selection criteria for efficient adversarial training, significantly reducing computation without sacrificing robustness.

Findings

Achieves comparable or better robustness than full PGD training.

Reduces adversarial computation by up to 50%.

Effective on MNIST and CIFAR-10 datasets.

Abstract

Projected Gradient Descent (PGD) is a strong and widely used first-order adversarial attack, yet its computational cost scales poorly, as all training samples undergo identical iterative inner-loop optimization despite contributing unequally to robustness. Motivated by this inefficiency, we propose \emph{Selective Adversarial Training}, which perturbs only a subset of critical samples in each minibatch. Specifically, we introduce two principled selection criteria: (1) margin-based sampling, which prioritizes samples near the decision boundary, and (2) gradient-matching sampling, which selects samples whose gradients align with the dominant batch optimization direction. Adversarial examples are generated only for the selected subset, while the remaining samples are trained cleanly using a mixed objective. Experiments on MNIST and CIFAR-10 show that the proposed methods achieve robustness comparable to, or even exceeding, full PGD adversarial training, while reducing adversarial computation by up to $50\%$, demonstrating that informed sample selection is sufficient for scalable adversarial robustness.