An Efficient Evolutionary Deep Learning Framework Based on Multi-source Transfer Learning to Evolve Deep Convolutional Neural Networks

TL;DR

This paper introduces an efficient evolutionary deep learning framework that utilizes multi-source transfer learning to rapidly evolve high-performing CNNs with reduced computational costs.

Contribution

It proposes a novel transfer learning-inspired evolutionary framework that leverages multi-source domains to efficiently evolve CNN blocks and optimize their architecture.

Findings

Achieves faster CNN evolution within 40 GPU-hours compared to 15 peers.

Attains state-of-the-art error rates on CIFAR-10, CIFAR-100, and SVHN datasets.

Demonstrates strong competitiveness in classification accuracy with reduced computational resources.

Abstract

Convolutional neural networks (CNNs) have constantly achieved better performance over years by introducing more complex topology, and enlarging the capacity towards deeper and wider CNNs. This makes the manual design of CNNs extremely difficult, so the automated design of CNNs has come into the research spotlight, which has obtained CNNs that outperform manually-designed CNNs. However, the computational cost is still the bottleneck of automatically designing CNNs. In this paper, inspired by transfer learning, a new evolutionary computation based framework is proposed to efficiently evolve CNNs without compromising the classification accuracy. The proposed framework leverages multi-source domains, which are smaller datasets than the target domain datasets, to evolve a generalised CNN block only once. And then, a new stacking method is proposed to both widen and deepen the evolved block, and a grid search method is proposed to find optimal stacking solutions. The experimental results show the proposed method acquires good CNNs faster than 15 peer competitors within less than 40 GPU-hours. Regarding the classification accuracy, the proposed method gains its strong competitiveness against the peer competitors, which achieves the best error rates of 3.46%, 18.36% and 1.76% for the CIFAR-10, CIFAR-100 and SVHN datasets, respectively.