Evolution of Convolutional Highway Networks

TL;DR

This paper presents an evolutionary algorithm to optimize convolutional highway networks, demonstrating its ability to improve state-of-the-art performance on MNIST by evolving networks from scratch.

Contribution

It introduces an EA-based method for optimizing the structure and hyperparameters of convolutional highway networks, a novel approach in this context.

Findings

EA improves state-of-the-art network performance

EA can evolve networks from scratch

Method effectively overcomes local optima

Abstract

Convolutional highways are deep networks based on multiple stacked convolutional layers for feature preprocessing. We introduce an evolutionary algorithm (EA) for optimization of the structure and hyperparameters of convolutional highways and demonstrate the potential of this optimization setting on the well-known MNIST data set. The (1+1)-EA employs Rechenberg's mutation rate control and a niching mechanism to overcome local optima adapts the optimization approach. An experimental study shows that the EA is capable of improving the state-of-the-art network contribution and of evolving highway networks from scratch.