Hybrid Genetic Algorithm and Hill Climbing Optimization for the Neural Network

TL;DR

This paper introduces a hybrid genetic algorithm and hill climbing method to optimize CNN hyperparameters, achieving better accuracy with fewer generations on CIFAR-100.

Contribution

It presents a novel hybrid optimization approach combining global and local search techniques for CNN hyperparameter tuning.

Findings

Hybrid model outperforms standard algorithms in accuracy.

Fewer generations needed for optimal results.

Effective balance of exploration and exploitation.

Abstract

In this paper, we propose a hybrid model combining genetic algorithm and hill climbing algorithm for optimizing Convolutional Neural Networks (CNNs) on the CIFAR-100 dataset. The proposed model utilizes a population of chromosomes that represent the hyperparameters of the CNN model. The genetic algorithm is used for selecting and breeding the fittest chromosomes to generate new offspring. The hill climbing algorithm is then applied to the offspring to further optimize their hyperparameters. The mutation operation is introduced to diversify the population and to prevent the algorithm from getting stuck in local optima. The Genetic Algorithm is used for global search and exploration of the search space, while Hill Climbing is used for local optimization of promising solutions. The objective function is the accuracy of the trained neural network on the CIFAR-100 test set. The performance of the hybrid model is evaluated by comparing it with the standard genetic algorithm and hill-climbing algorithm. The experimental results demonstrate that the proposed hybrid model achieves better accuracy with fewer generations compared to the standard algorithms. Therefore, the proposed hybrid model can be a promising approach for optimizing CNN models on large datasets.