Robust Neural Architecture Search

TL;DR

This paper introduces Robust Neural Architecture Search (RNAS), a method that balances accuracy and robustness in neural network architectures, reducing search costs by using noise instead of adversarial examples, and achieving state-of-the-art results.

Contribution

RNAS is a novel NAS approach that incorporates a regularization term for robustness and uses noise inputs to efficiently search for architectures with high accuracy and robustness.

Findings

RNAS achieves SOTA performance on image classification.

RNAS provides a good tradeoff between robustness and accuracy.

Using noise examples reduces search cost compared to adversarial examples.

Abstract

Neural Architectures Search (NAS) becomes more and more popular over these years. However, NAS-generated models tends to suffer greater vulnerability to various malicious attacks. Lots of robust NAS methods leverage adversarial training to enhance the robustness of NAS-generated models, however, they neglected the nature accuracy of NAS-generated models. In our paper, we propose a novel NAS method, Robust Neural Architecture Search (RNAS). To design a regularization term to balance accuracy and robustness, RNAS generates architectures with both high accuracy and good robustness. To reduce search cost, we further propose to use noise examples instead adversarial examples as input to search architectures. Extensive experiments show that RNAS achieves state-of-the-art (SOTA) performance on both image classification and adversarial attacks, which illustrates the proposed RNAS achieves a good tradeoff between robustness and accuracy.