A Survey on Evolutionary Neural Architecture Search

TL;DR

This survey comprehensively reviews over 200 evolutionary computation-based neural architecture search methods, discussing their design principles, challenges, and future directions in automating neural network architecture design.

Contribution

It provides the first systematic summary of EC-based NAS algorithms, analyzing their core components and design justifications.

Findings

EC-based NAS methods have gained significant attention and success.

Current challenges include scalability and search efficiency.

Future research directions involve addressing these challenges and improving algorithm robustness.

Abstract

Deep Neural Networks (DNNs) have achieved great success in many applications. The architectures of DNNs play a crucial role in their performance, which is usually manually designed with rich expertise. However, such a design process is labour intensive because of the trial-and-error process, and also not easy to realize due to the rare expertise in practice. Neural Architecture Search (NAS) is a type of technology that can design the architectures automatically. Among different methods to realize NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. Unfortunately, there has not yet been a comprehensive summary of the EC-based NAS algorithms. This paper reviews over 200 papers of most recent EC-based NAS methods in light of the core components, to systematically discuss their design principles as well as justifications on the design. Furthermore, current challenges and issues are also discussed to identify future research in this emerging field.