Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search

TL;DR

This paper introduces a continuous Cartesian genetic programming approach for neural architecture search, optimizing CNNs with multi-objective evolutionary algorithms to balance performance and complexity.

Contribution

It presents a novel CGP-based representation for NAS, combining real-based and block-chained CNNs, and evaluates multiple MOEAs on CIFAR datasets.

Findings

Competitive classification accuracy achieved.

Reduced model complexity compared to state-of-the-art.

Effective search space exploration with proposed methods.

Abstract

We propose a novel approach for the challenge of designing less complex yet highly effective convolutional neural networks (CNNs) through the use of cartesian genetic programming (CGP) for neural architecture search (NAS). Our approach combines real-based and block-chained CNNs representations based on CGP for optimization in the continuous domain using multi-objective evolutionary algorithms (MOEAs). Two variants are introduced that differ in the granularity of the search space they consider. The proposed CGP-NASV1 and CGP-NASV2 algorithms were evaluated using the non-dominated sorting genetic algorithm II (NSGA-II) on the CIFAR-10 and CIFAR-100 datasets. The empirical analysis was extended to assess the crossover operator from differential evolution (DE), the multi-objective evolutionary algorithm based on decomposition (MOEA/D) and S metric selection evolutionary multi-objective algorithm (SMS-EMOA) using the same representation. Experimental results demonstrate that our approach is competitive with state-of-the-art proposals in terms of classification performance and model complexity.