AdvRush: Searching for Adversarially Robust Neural Architectures

TL;DR

AdvRush introduces a neural architecture search method that enhances intrinsic adversarial robustness by optimizing for smoother input loss landscapes, leading to more resilient neural networks.

Contribution

This work presents AdvRush, a novel architecture search algorithm that improves robustness by focusing on the intrinsic properties of the network's loss landscape, beyond traditional training methods.

Findings

Achieves 55.91% robust accuracy on CIFAR-10 under FGSM attack after standard training.

Attains 50.04% robust accuracy under AutoAttack after PGD adversarial training.

Demonstrates effectiveness across various benchmark datasets.

Abstract

Deep neural networks continue to awe the world with their remarkable performance. Their predictions, however, are prone to be corrupted by adversarial examples that are imperceptible to humans. Current efforts to improve the robustness of neural networks against adversarial examples are focused on developing robust training methods, which update the weights of a neural network in a more robust direction. In this work, we take a step beyond training of the weight parameters and consider the problem of designing an adversarially robust neural architecture with high intrinsic robustness. We propose AdvRush, a novel adversarial robustness-aware neural architecture search algorithm, based upon a finding that independent of the training method, the intrinsic robustness of a neural network can be represented with the smoothness of its input loss landscape. Through a regularizer that favors a candidate architecture with a smoother input loss landscape, AdvRush successfully discovers an adversarially robust neural architecture. Along with a comprehensive theoretical motivation for AdvRush, we conduct an extensive amount of experiments to demonstrate the efficacy of AdvRush on various benchmark datasets. Notably, on CIFAR-10, AdvRush achieves 55.91% robust accuracy under FGSM attack after standard training and 50.04% robust accuracy under AutoAttack after 7-step PGD adversarial training.