Explaining the role of Intrinsic Dimensionality in Adversarial Training

TL;DR

This paper explains how intrinsic dimensionality influences adversarial training outcomes across different models, proposing a new scalable method that enhances robustness while maintaining generalization.

Contribution

It introduces the manifold conjecture-based explanation for model behavior and proposes SMAAT, a novel scalable adversarial training method leveraging intrinsic dimensionality.

Findings

SMAAT reduces GPU time by 25-33% during adversarial training.

Models with low intrinsic dimensionality in certain layers show improved robustness.

SMAAT achieves superior robustness across multiple tasks with comparable generalization.

Abstract

Adversarial Training (AT) impacts different architectures in distinct ways: vision models gain robustness but face reduced generalization, encoder-based models exhibit limited robustness improvements with minimal generalization loss, and recent work in latent-space adversarial training (LAT) demonstrates that decoder-based models achieve improved robustness by applying AT across multiple layers. We provide the first explanation for these trends by leveraging the manifold conjecture: off-manifold adversarial examples (AEs) enhance robustness, while on-manifold AEs improve generalization. We show that vision and decoder-based models exhibit low intrinsic dimensionality in earlier layers (favoring off-manifold AEs), whereas encoder-based models do so in later layers (favoring on-manifold AEs). Exploiting this property, we introduce SMAAT, which improves the scalability of AT for encoder-based models by perturbing the layer with the lowest intrinsic dimensionality. This reduces the projected gradient descent (PGD) chain length required for AE generation, cutting GPU time by 25-33% while significantly boosting robustness. We validate SMAAT across multiple tasks, including text generation, sentiment classification, safety filtering, and retrieval augmented generation setups, demonstrating superior robustness with comparable generalization to standard training.