Progressive Neural Architecture Search
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua,, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy

TL;DR
This paper introduces a sequential model-based optimization approach for neural architecture search that significantly improves efficiency over previous reinforcement learning methods, leading to state-of-the-art CNN structures.
Contribution
It presents a novel SMBO-based method for neural architecture search that is more efficient and effective than existing RL and evolutionary approaches.
Findings
Up to 5x fewer models evaluated compared to RL methods
Achieves state-of-the-art accuracy on CIFAR-10 and ImageNet
8x faster in total compute time than previous methods
Abstract
We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space. Direct comparison under the same search space shows that our method is up to 5 times more efficient than the RL method of Zoph et al. (2018) in terms of number of models evaluated, and 8 times faster in terms of total compute. The structures we discover in this way achieve state of the art classification accuracies on CIFAR-10 and ImageNet.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Machine Learning and Data Classification · Adversarial Robustness in Machine Learning
MethodsDense Connections · Feedforward Network · Softmax · Depthwise Convolution · Pointwise Convolution · Depthwise Separable Convolution · Convolution · Max Pooling · RMSProp · Progressive Neural Architecture Search
