Evolutionary NAS with Gene Expression Programming of Cellular Encoding

TL;DR

This paper introduces a new encoding scheme called SLGE for evolutionary neural architecture search, enabling scalable CNN design that outperforms handcrafted architectures and is resource-efficient.

Contribution

The paper proposes SLGE, a novel linear encoding scheme for evolutionary NAS that improves scalability and performance over existing methods.

Findings

SLGE effectively discovers CNN architectures that outperform handcrafted models on CIFAR datasets.

SLGE achieves competitive error rates with less GPU resources compared to other NAS methods.

The encoding scheme simplifies the evolutionary process and enhances scalability for large architectures.

Abstract

The renaissance of neural architecture search (NAS) has seen classical methods such as genetic algorithms (GA) and genetic programming (GP) being exploited for convolutional neural network (CNN) architectures. While recent work have achieved promising performance on visual perception tasks, the direct encoding scheme of both GA and GP has functional complexity deficiency and does not scale well on large architectures like CNN. To address this, we present a new generative encoding scheme -- $symbolic\ linear\ generative\ encoding$ (SLGE) -- simple, yet powerful scheme which embeds local graph transformations in chromosomes of linear fixed-length string to develop CNN architectures of variant shapes and sizes via evolutionary process of gene expression programming. In experiments, the effectiveness of SLGE is shown in discovering architectures that improve the performance of the state-of-the-art handcrafted CNN architectures on CIFAR-10 and CIFAR-100 image classification tasks; and achieves a competitive classification error rate with the existing NAS methods using less GPU resources.