On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future

TL;DR

This paper introduces a method to leverage residual information from neuroevolutionary searches, using a Bayesian network to improve future neural architecture searches and reduce computational costs.

Contribution

It proposes a novel approach to extract valuable knowledge from neuroevolutionary runs to enhance and accelerate future neural network design processes.

Findings

Bayesian network effectively models neuroevolutionary search data

The approach can identify strong neural structures immediately

It helps initialize and guide future neural architecture searches

Abstract

Neuroevolutionary algorithms, automatic searches of neural network structures by means of evolutionary techniques, are computationally costly procedures. In spite of this, due to the great performance provided by the architectures which are found, these methods are widely applied. The final outcome of neuroevolutionary processes is the best structure found during the search, and the rest of the procedure is commonly omitted in the literature. However, a good amount of residual information consisting of valuable knowledge that can be extracted is also produced during these searches. In this paper, we propose an approach that extracts this information from neuroevolutionary runs, and use it to build a metamodel that could positively impact future neural architecture searches. More specifically, by inspecting the best structures found during neuroevolutionary searches of generative adversarial networks with varying characteristics (e.g., based on dense or convolutional layers), we propose a Bayesian network-based model which can be used to either find strong neural structures right away, conveniently initialize different structural searches for different problems, or help future optimization of structures of any type to keep finding increasingly better structures where uninformed methods get stuck into local optima.