Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

TL;DR

This paper provides a comprehensive survey of neuroevolution methods, specifically evolutionary algorithms, for configuring and training deep neural networks, highlighting current challenges and future research directions.

Contribution

It offers the first extensive review focused solely on neuroevolution in DNNs, analyzing strengths, limitations, and potential for future advancements.

Findings

Neuroevolution is gaining momentum for DNN optimization.

Current challenges include scalability and computational cost.

Future directions involve hybrid methods and automated design techniques.

Abstract

A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. Neuroevolution is a term which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Prolonged absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for improving performance and wider application within real-world deep learning problems. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs. Based on this survey, the paper highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions.