Fast-DENSER++: Evolving Fully-Trained Deep Artificial Neural Networks

TL;DR

Fast-DENSER++ is a novel neuroevolution method that evolves fully-trained deep neural networks by adaptively increasing training time during evolution, resulting in better-performing models ready for deployment without additional training.

Contribution

It introduces a dynamic training time mechanism in neuroevolution, enabling the evolution of fully-trained DANNs, which was not addressed in prior fixed-epoch evaluation methods.

Findings

F-DENSER++ produces fully-trained DANNs with an average accuracy of 88.73%.

It outperforms previous DENSER (Fast-DENSER) with statistically significant improvements.

Longer training cycles during evolution enhance model performance.

Abstract

This paper proposes a new extension to Deep Evolutionary Network Structured Evolution (DENSER), called Fast-DENSER++ (F-DENSER++). The vast majority of NeuroEvolution methods that optimise Deep Artificial Neural Networks (DANNs) only evaluate the candidate solutions for a fixed amount of epochs; this makes it difficult to effectively assess the learning strategy, and requires the best generated network to be further trained after evolution. F-DENSER++ enables the training time of the candidate solutions to grow continuously as necessary, i.e., in the initial generations the candidate solutions are trained for shorter times, and as generations proceed it is expected that longer training cycles enable better performances. Consequently, the models discovered by F-DENSER++ are fully-trained DANNs, and are ready for deployment after evolution, without the need for further training. The results demonstrate the ability of F-DENSER++ to effectively generate fully-trained DANNs; by the end of evolution, whilst the average performance of the models generated by F-DENSER++ is of 88.73%, the performance of the models generated by the previous version of DENSER (Fast-DENSER) is 86.91% (statistically significant), which increases to 87.76% when allowed to train for longer.