Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS

TL;DR

This paper compares grid search, random search, and genetic algorithms for neural architecture search, evaluating their efficiency and accuracy on CIFAR-10 to identify the most effective hyperparameter optimization method.

Contribution

It provides a systematic comparison of three popular hyperparameter optimization algorithms applied to neural architecture search, highlighting their performance differences.

Findings

Genetic Algorithm achieves higher accuracy

Random Search is faster than Grid Search

Performance varies significantly across algorithms

Abstract

In this paper, we compare the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempt to use them for neural architecture search (NAS). We use these algorithms for building a convolutional neural network (search architecture). Experimental results on CIFAR-10 dataset further demonstrate the performance difference between compared algorithms. The comparison results are based on the execution time of the above algorithms and accuracy of the proposed models.