FNAS: Uncertainty-Aware Fast Neural Architecture Search

TL;DR

This paper introduces FNAS, a novel neural architecture search method that significantly accelerates RL-based NAS by leveraging architecture and parameter transfer, uncertainty-aware critics, and block-level weight sharing, achieving about 10x speedup.

Contribution

FNAS is the first to incorporate block-level weight sharing in RL-based NAS and introduces an uncertainty-aware critic and a knowledge pool to drastically reduce search time.

Findings

FNAS accelerates RL-based NAS by approximately 10 times.

The method achieves better performance on various vision tasks.

It reduces search time from 20,000 GPU hours to 2,000 GPU hours.

Abstract

Reinforcement learning (RL)-based neural architecture search (NAS) generally guarantees better convergence yet suffers from the requirement of huge computational resources compared with gradient-based approaches, due to the rollout bottleneck -- exhaustive training for each sampled generation on proxy tasks. In this paper, we propose a general pipeline to accelerate the convergence of the rollout process as well as the RL process in NAS. It is motivated by the interesting observation that both the architecture and the parameter knowledge can be transferred between different experiments and even different tasks. We first introduce an uncertainty-aware critic (value function) in Proximal Policy Optimization (PPO) to utilize the architecture knowledge in previous experiments, which stabilizes the training process and reduces the searching time by 4 times. Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times. It is the first to introduce block-level weight sharing in RLbased NAS. The block similarity function guarantees a 100% hitting ratio with strict fairness. Besides, we show that a simply designed off-policy correction factor used in "replay buffer" in RL optimization can further reduce half of the searching time. Experiments on the Mobile Neural Architecture Search (MNAS) search space show the proposed Fast Neural Architecture Search (FNAS) accelerates standard RL-based NAS process by ~10x (e.g. ~256 2x2 TPUv2 x days / 20,000 GPU x hour -> 2,000 GPU x hour for MNAS), and guarantees better performance on various vision tasks.