RATs-NAS: Redirection of Adjacent Trails on GCN for Neural Architecture Search

TL;DR

RATs-NAS introduces a novel GCN-based predictor and a sampling method to accelerate neural architecture search, achieving superior results on NASBench-201 datasets.

Contribution

The paper proposes RATs-NAS, combining RATs-GCN and P3S modules, to efficiently explore neural architectures with improved accuracy and speed over existing NAS methods.

Findings

RATs-NAS outperforms WeakNAS and Arch-Graph on NASBench-201.

The method effectively narrows search space intervals based on FLOPs.

RATs-NAS demonstrates significant margin improvements in architecture search results.

Abstract

Various hand-designed CNN architectures have been developed, such as VGG, ResNet, DenseNet, etc., and achieve State-of-the-Art (SoTA) levels on different tasks. Neural Architecture Search (NAS) now focuses on automatically finding the best CNN architecture to handle the above tasks. However, the verification of a searched architecture is very time-consuming and makes predictor-based methods become an essential and important branch of NAS. Two commonly used techniques to build predictors are graph-convolution networks (GCN) and multilayer perceptron (MLP). In this paper, we consider the difference between GCN and MLP on adjacent operation trails and then propose the Redirected Adjacent Trails NAS (RATs-NAS) to quickly search for the desired neural network architecture. The RATs-NAS consists of two components: the Redirected Adjacent Trails GCN (RATs-GCN) and the Predictor-based Search Space Sampling (P3S) module. RATs-GCN can change trails and their strengths to search for a better neural network architecture. P3S can rapidly focus on tighter intervals of FLOPs in the search space. Based on our observations on cell-based NAS, we believe that architectures with similar FLOPs will perform similarly. Finally, the RATs-NAS consisting of RATs-GCN and P3S beats WeakNAS, Arch-Graph, and others by a significant margin on three sub-datasets of NASBench-201.