Evolving Deep Neural Networks

TL;DR

This paper introduces CoDeepNEAT, an evolutionary algorithm that automates the design of deep neural network architectures, achieving competitive results in benchmarks and enabling real-world applications like automated image captioning.

Contribution

It extends neuroevolution methods to optimize topology, components, and hyperparameters of deep networks, reducing manual design effort.

Findings

Achieves results comparable to human-designed architectures in benchmarks.

Supports real-world application of automated image captioning.

Demonstrates potential for future deep learning architecture automation.

Abstract

The success of deep learning depends on finding an architecture to fit the task. As deep learning has scaled up to more challenging tasks, the architectures have become difficult to design by hand. This paper proposes an automated method, CoDeepNEAT, for optimizing deep learning architectures through evolution. By extending existing neuroevolution methods to topology, components, and hyperparameters, this method achieves results comparable to best human designs in standard benchmarks in object recognition and language modeling. It also supports building a real-world application of automated image captioning on a magazine website. Given the anticipated increases in available computing power, evolution of deep networks is promising approach to constructing deep learning applications in the future.