Towards co-evolution of fitness predictors and Deep Neural Networks

TL;DR

This paper proposes a method to accelerate neural network architecture evolution by using fitness prediction based on training subsets, aiming to reduce evaluation time and automate topology design.

Contribution

It introduces a fitness prediction approach using training subsets to estimate network performance, facilitating faster evolutionary topology search.

Findings

Subsets can approximate full training fitness effectively.

Fitness prediction reduces evaluation time significantly.

Feasibility depends on subset selection quality.

Abstract

Deep neural networks proved to be a very useful and powerful tool with many practical applications. They especially excel at learning from large data sets with labeled samples. However, in order to achieve good learning results, the network architecture has to be carefully designed. Creating an optimal topology requires a lot of experience and knowledge. Unfortunately there are no practically applicable algorithms which could help in this situation. Using an evolutionary process to develop new network topologies might solve this problem. The limiting factor in this case is the speed of evaluation of a single specimen (a single network architecture), which includes learning based on the whole large dataset. In this paper we propose to overcome this problem by using a fitness prediction technique: use subsets of the original training set to conduct the training process and use its results as an approximation of specimen's fitness. We discuss the feasibility of this approach in context of the desired fitness predictor features and analyze whether subsets obtained in an evolutionary process can be used to estimate the fitness of the network topology. Finally we draw conclusions from our experiments and outline plans for future work.