L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning

TL;DR

L$^{2}$NAS introduces a reinforcement learning approach to neural architecture search that learns to optimize hyperparameters efficiently, achieving state-of-the-art results and demonstrating transferability across datasets.

Contribution

The paper presents L$^{2}$NAS, a novel actor-critic reinforcement learning method for neural architecture search that improves over gradient-based approaches.

Findings

Achieves state-of-the-art results on NAS-Bench-201 and other search spaces.

Demonstrates transferability of search policies across datasets.

Efficient training via a quantile-driven actor-critic framework.

Abstract

Neural architecture search (NAS) has achieved remarkable results in deep neural network design. Differentiable architecture search converts the search over discrete architectures into a hyperparameter optimization problem which can be solved by gradient descent. However, questions have been raised regarding the effectiveness and generalizability of gradient methods for solving non-convex architecture hyperparameter optimization problems. In this paper, we propose L$^{2}$NAS, which learns to intelligently optimize and update architecture hyperparameters via an actor neural network based on the distribution of high-performing architectures in the search history. We introduce a quantile-driven training procedure which efficiently trains L$^{2}$NAS in an actor-critic framework via continuous-action reinforcement learning. Experiments show that L$^{2}$NAS achieves state-of-the-art results on NAS-Bench-201 benchmark as well as DARTS search space and Once-for-All MobileNetV3 search space. We also show that search policies generated by L$^{2}$NAS are generalizable and transferable across different training datasets with minimal fine-tuning.