Taming the Long Tail: Rebalancing Adversarial Training via Adaptive Perturbation

TL;DR

This paper introduces RobustLT, a framework that adaptively adjusts adversarial perturbations during training to improve robustness and class balance in long-tail datasets.

Contribution

It provides a theoretical analysis of adversarial training on long-tail data and proposes a novel adaptive perturbation method to address class imbalance and vulnerability.

Findings

RobustLT improves adversarial robustness on long-tailed datasets.

It enhances class balance during adversarial training.

The framework is effective across multiple datasets.

Abstract

Deep neural networks are highly vulnerable to adversarial examples, i.e.,small perturbations that can significantly degrade model performance. While adversarial training has become the primary defense strategy, most studies focus on balanced datasets, overlooking the challenges posed by real-world long-tail data. Motivated by the fact that perturbations in adversarial examples inherently alter the training distribution, we theoretically investigate their impact. We first revisit adversarial training for long-tail data and identify two key limitations: (i) a skewed training objective caused by class imbalance, and (ii) unstable evolution of adversarial distributions. Furthermore, we show that perturbations can simultaneously address both adversarial vulnerability and class imbalance. Based on these insights, we propose RobustLT, a plug-and-play framework that adaptively adjusts perturbations during adversarial training. Extensive experiments demonstrate that RobustLT consistently enhances adversarial robustness and class-balance on long-tailed datasets. The code is available at \href{https://github.com/zhang-lilin/RobustLT}{https://github.com/zhang-lilin/RobustLT}.