Neural Architecture Search: A Survey

TL;DR

This survey reviews the field of neural architecture search, highlighting its importance in automating the design of neural networks to improve efficiency and performance across various tasks.

Contribution

It categorizes existing neural architecture search methods based on search space, strategy, and performance estimation, providing a comprehensive overview of the field.

Findings

Neural architecture search automates network design, reducing manual effort.

Various search strategies and performance estimation methods exist.

The survey organizes the field into a clear taxonomy.

Abstract

Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, which is a time-consuming and error-prone process. Because of this, there is growing interest in automated neural architecture search methods. We provide an overview of existing work in this field of research and categorize them according to three dimensions: search space, search strategy, and performance estimation strategy.