Trends in Neural Architecture Search: Towards the Acceleration of Search

TL;DR

This paper reviews current trends in neural architecture search, classifying methods into neuro-evolutionary, reinforcement learning, and one-shot approaches, comparing them and discussing future directions.

Contribution

It provides a comprehensive classification and comparison of major NAS research trends and outlines future research directions.

Findings

Neuro-evolutionary algorithms are a key NAS trend.

Reinforcement learning based NAS methods are widely studied.

One-shot architecture search approaches offer promising efficiency.

Abstract

In modern deep learning research, finding optimal (or near optimal) neural network models is one of major research directions and it is widely studied in many applications. In this paper, the main research trends of neural architecture search (NAS) are classified as neuro-evolutionary algorithms, reinforcement learning based algorithms, and one-shot architecture search approaches. Furthermore, each research trend is introduced and finally all the major three trends are compared. Lastly, the future research directions of NAS research trends are discussed.