Discovering Robust Convolutional Architecture at Targeted Capacity: A Multi-Shot Approach

TL;DR

This paper introduces a multi-shot neural architecture search method to find adversarially robust CNN architectures at specific capacity targets, outperforming existing NAS-discovered models in robustness and accuracy.

Contribution

The paper proposes a novel multi-shot NAS approach that explicitly searches for robust architectures at targeted capacities, addressing limitations of one-shot NAS methods.

Findings

MSRobNet-2000 outperforms RobNet-large by 4%-7% in robustness.

MSRobNet-1560 surpasses RobNet-free by 2.3% in clean accuracy.

The method effectively discovers robust architectures at specified FLOPs targets.

Abstract

Convolutional neural networks (CNNs) are vulnerable to adversarial examples, and studies show that increasing the model capacity of an architecture topology (e.g., width expansion) can bring consistent robustness improvements. This reveals a clear robustness-efficiency trade-off that should be considered in architecture design. In this paper, considering scenarios with capacity budget, we aim to discover adversarially robust architecture at targeted capacities. Recent studies employed one-shot neural architecture search (NAS) to discover robust architectures. However, since the capacities of different topologies cannot be aligned in the search process, one-shot NAS methods favor topologies with larger capacities in the supernet. And the discovered topology might be suboptimal when augmented to the targeted capacity. We propose a novel multi-shot NAS method to address this issue and explicitly search for robust architectures at targeted capacities. At the targeted FLOPs of 2000M, the discovered MSRobNet-2000 outperforms the recent NAS-discovered architecture RobNet-large under various criteria by a large margin of 4%-7%. And at the targeted FLOPs of 1560M, MSRobNet-1560 surpasses another NAS-discovered architecture RobNet-free by 2.3% and 1.3% in the clean and PGD-7 accuracies, respectively. All codes are available at https://github.com/walkerning/aw\_nas.