Towards Deep Learning Models Resistant to Transfer-based Adversarial Attacks via Data-centric Robust Learning

TL;DR

This paper introduces Data-centric Robust Learning (DRL), a new efficient defense against transfer-based adversarial attacks that outperforms traditional adversarial training methods by using a one-shot data augmentation approach.

Contribution

The paper proposes DRL, a novel data-centric approach that reduces computational overhead and enhances robustness against transfer-based attacks compared to existing adversarial training methods.

Findings

DRL outperforms traditional adversarial training methods in robustness.

DRL surpasses top defenses on RobustBench when combined with data augmentation.

DRL improves model generalization and robust fairness.

Abstract

Transfer-based adversarial attacks raise a severe threat to real-world deep learning systems since they do not require access to target models. Adversarial training (AT), which is recognized as the strongest defense against white-box attacks, has also guaranteed high robustness to (black-box) transfer-based attacks. However, AT suffers from heavy computational overhead since it optimizes the adversarial examples during the whole training process. In this paper, we demonstrate that such heavy optimization is not necessary for AT against transfer-based attacks. Instead, a one-shot adversarial augmentation prior to training is sufficient, and we name this new defense paradigm Data-centric Robust Learning (DRL). Our experimental results show that DRL outperforms widely-used AT techniques (e.g., PGD-AT, TRADES, EAT, and FAT) in terms of black-box robustness and even surpasses the top-1 defense on RobustBench when combined with diverse data augmentations and loss regularizations. We also identify other benefits of DRL, for instance, the model generalization capability and robust fairness.