ASNN: Learning to Suggest Neural Architectures from Performance Distributions

TL;DR

This paper introduces ASNN, a neural network model that learns the relationship between architecture and accuracy to suggest improved designs, outperforming random search in automating neural network architecture optimization.

Contribution

The paper presents ASNN, a novel model that predicts and suggests neural architectures with higher performance, providing an efficient alternative to traditional search methods.

Findings

ASNN successfully suggested architectures with higher test accuracy.

Repeated cycles of prediction and retraining improved architecture performance.

ASNN outperformed random search in architecture optimization.

Abstract

The architecture of a neural network (NN) plays a critical role in determining its performance. However, there is no general closed-form function that maps between network structure and accuracy, making the process of architecture design largely heuristic or search-based. In this study, we propose the Architecture Suggesting Neural Network (ASNN), a model designed to learn the relationship between NN architecture and its test accuracy, and to suggest improved architectures accordingly. To train ASNN, we constructed datasets using TensorFlow-based models with varying numbers of layers and nodes. Experimental results were collected for both 2-layer and 3-layer architectures across a grid of configurations, each evaluated with 10 repeated trials to account for stochasticity. Accuracy values were treated as inputs, and architectural parameters as outputs. The trained ASNN was then used iteratively to predict architectures that yield higher performance. In both 2-layer and 3-layer cases, ASNN successfully suggested architectures that outperformed the best results found in the original training data. Repeated prediction and retraining cycles led to the discovery of architectures with improved mean test accuracies, demonstrating the model's capacity to generalize the performance-structure relationship. These results suggest that ASNN provides an efficient alternative to random search for architecture optimization, and offers a promising approach toward automating neural network design. "Parts of the manuscript, including text editing and expression refinement, were supported by OpenAI's ChatGPT. All content was reviewed and verified by the authors."