A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS

TL;DR

This paper introduces GATES, a graph-based encoding scheme for neural architecture search that models data processing more realistically, enhancing predictor accuracy and search efficiency across diverse search spaces.

Contribution

GATES provides a novel, more accurate graph-based encoding for neural architectures, improving predictor performance and sample efficiency in NAS.

Findings

GATES outperforms existing encoding schemes in predictor accuracy.

Enhanced predictor leads to more efficient neural architecture search.

Effective across multiple search spaces.

Abstract

This work proposes a novel Graph-based neural ArchiTecture Encoding Scheme, a.k.a. GATES, to improve the predictor-based neural architecture search. Specifically, different from existing graph-based schemes, GATES models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture. GATES is a more reasonable modeling of the neural architectures, and can encode architectures from both the "operation on node" and "operation on edge" cell search spaces consistently. Experimental results on various search spaces confirm GATES's effectiveness in improving the performance predictor. Furthermore, equipped with the improved performance predictor, the sample efficiency of the predictor-based neural architecture search (NAS) flow is boosted. Codes are available at https://github.com/walkerning/aw_nas.