Designing Neural Network Architectures using Reinforcement Learning

TL;DR

This paper presents MetaQNN, a reinforcement learning-based method that automatically designs CNN architectures, outperforming handcrafted models and existing meta-modeling approaches on image classification benchmarks.

Contribution

Introduction of MetaQNN, a reinforcement learning algorithm that automatically generates high-performing CNN architectures, reducing reliance on human expertise and manual experimentation.

Findings

MetaQNN-designed networks outperform handcrafted CNNs.

The method is competitive with state-of-the-art complex models.

MetaQNN outperforms existing meta-modeling approaches.

Abstract

At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $\epsilon$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.