# Evolutionary bi-level neural architecture search with training: A framework for color classification

## TL;DR

This paper introduces a new method for designing efficient neural networks that achieve high performance while using fewer resources.

## Contribution

The novel EB-LNAST framework simultaneously optimizes architecture, weights, and biases using bi-level optimization.

## Key findings

- EB-LNAST outperforms traditional machine learning algorithms and advanced models in predictive performance.
- It achieves up to 99.66% reduction in model size compared to MLPs with hyperparameter tuning.
- The method remains competitive with only a 0.99% performance reduction when compared to optimized MLPs.

## Abstract

The design of Artificial Neural Networks (ANNs) for classification tasks has been a topic of interest. However, defining an optimal ANN architecture remains challenging, especially when considering resource constraints and the large number of design parameters. This paper proposes an Evolutionary Bi-Level Neural Architecture Search with Training (EB-LNAST) approach that simultaneously optimizes the architecture, weights, and biases of a neural network using a bi-level optimization strategy. The upper level focuses on minimizing the network complexity penalized by the lower level performance function, while the lower level optimizes training parameters to minimize the loss function and maximize the predictive performance. The proposal is evaluated on a real-world color classification task and the WDBC dataset, demonstrating statistically significant improvements over traditional machine learning algorithms, as well as advanced models. Compared to Multilayer Perceptron (MLP) based algorithms, EB-LNAST achieves superior predictive performance when the architecture is fixed, and remains competitive, with a marginal reduction in performance of no more than \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$0.99\%$$\end{document}, even when compared against MLPs optimized with extensive hyperparameter tuning, including architecture, activation functions, regularization, and optimizers. Remarkably, EB-LNAST achieves up to a \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$99.66\%$$\end{document} reduction in model size, highlighting its ability to discover compact and efficient architectures. EB-LNAST is a reliable alternative for generating compact and effective neural network architectures in accordance with the problem’s requirements, enabling efficient exploration of the search space while maintaining or exceeding the predictive performance of state-of-the-art classification algorithms.