RobArch: Designing Robust Architectures against Adversarial Attacks

TL;DR

This paper systematically studies how different architecture components affect the robustness of deep neural networks against adversarial attacks, leading to new design guidelines and a state-of-the-art robust model.

Contribution

It introduces the first large-scale analysis of architecture impacts on robustness and proposes RobArch, a model built on new guidelines achieving top adversarial accuracy.

Findings

18 actionable robust network design guidelines

RobArch achieves state-of-the-art AutoAttack accuracy on RobustBench ImageNet

Demonstrates architecture choices significantly influence adversarial robustness

Abstract

Adversarial Training is the most effective approach for improving the robustness of Deep Neural Networks (DNNs). However, compared to the large body of research in optimizing the adversarial training process, there are few investigations into how architecture components affect robustness, and they rarely constrain model capacity. Thus, it is unclear where robustness precisely comes from. In this work, we present the first large-scale systematic study on the robustness of DNN architecture components under fixed parameter budgets. Through our investigation, we distill 18 actionable robust network design guidelines that empower model developers to gain deep insights. We demonstrate these guidelines' effectiveness by introducing the novel Robust Architecture (RobArch) model that instantiates the guidelines to build a family of top-performing models across parameter capacities against strong adversarial attacks. RobArch achieves the new state-of-the-art AutoAttack accuracy on the RobustBench ImageNet leaderboard. The code is available at $\href{https://github.com/ShengYun-Peng/RobArch}{\text{this url}}$.