Concurrent Neural Tree and Data Preprocessing AutoML for Image Classification

TL;DR

This paper introduces a novel AutoML approach that simultaneously evolves neural network architectures and data preprocessing methods, improving image classification performance on CIFAR-10.

Contribution

It extends the EMADE framework to include data preprocessing primitives in neural architecture search, a novel integration not present in prior AutoML methods.

Findings

Improved CIFAR-10 classification accuracy

Effective joint evolution of data preprocessing and neural architectures

Potential for enhanced AutoML performance

Abstract

Deep Neural Networks (DNN's) are a widely-used solution for a variety of machine learning problems. However, it is often necessary to invest a significant amount of a data scientist's time to pre-process input data, test different neural network architectures, and tune hyper-parameters for optimal performance. Automated machine learning (autoML) methods automatically search the architecture and hyper-parameter space for optimal neural networks. However, current state-of-the-art (SOTA) methods do not include traditional methods for manipulating input data as part of the algorithmic search space. We adapt the Evolutionary Multi-objective Algorithm Design Engine (EMADE), a multi-objective evolutionary search framework for traditional machine learning methods, to perform neural architecture search. We also integrate EMADE's signal processing and image processing primitives. These primitives allow EMADE to manipulate input data before ingestion into the simultaneously evolved DNN. We show that including these methods as part of the search space shows potential to provide benefits to performance on the CIFAR-10 image classification benchmark dataset.