Ignition Phase : Standard Training for Fast Adversarial Robustness

TL;DR

This paper introduces Adversarial Evolution Training (AET), a framework that enhances adversarial robustness by adding an initial ERM phase, leading to faster, more effective training and improved accuracy across various datasets and models.

Contribution

The paper proposes AET, a novel training framework that preconditions feature representations with standard ERM training before adversarial training, improving efficiency and robustness.

Findings

AET achieves comparable or better robustness faster.

It improves clean accuracy and reduces training costs by 8-25%.

Effective across multiple datasets and architectures.

Abstract

Adversarial Training (AT) is a cornerstone defense, but many variants overlook foundational feature representations by primarily focusing on stronger attack generation. We introduce Adversarial Evolution Training (AET), a simple yet powerful framework that strategically prepends an Empirical Risk Minimization (ERM) phase to conventional AT. We hypothesize this initial ERM phase cultivates a favorable feature manifold, enabling more efficient and effective robustness acquisition. Empirically, AET achieves comparable or superior robustness more rapidly, improves clean accuracy, and cuts training costs by 8-25\%. Its effectiveness is shown across multiple datasets, architectures, and when augmenting established AT methods. Our findings underscore the impact of feature pre-conditioning via standard training for developing more efficient, principled robust defenses. Code is available in the supplementary material.