When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

TL;DR

TNAS introduces a tree-based NAS method that efficiently explores a reduced search space, achieving high accuracy and finding the global optimal architecture on CIFAR-10 in just four GPU hours.

Contribution

The paper presents a novel tree-based NAS approach that significantly reduces search complexity while improving accuracy over existing methods.

Findings

Achieves 94.37% accuracy on CIFAR-10 in four GPU hours.

Outperforms state-of-the-art NAS methods in search efficiency and accuracy.

Successfully finds the global optimal architecture on CIFAR-10.

Abstract

The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37\% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35\%, which outperforms the state-of-the-art. Code is available at: \url{https://github.com/guochengqian/TNAS}.