OptABC: an Optimal Hyperparameter Tuning Approach for Machine Learning Algorithms

TL;DR

This paper introduces OptABC, a novel hyperparameter tuning method combining swarm intelligence, clustering, and opposition strategies to improve convergence speed and accuracy in machine learning models.

Contribution

The paper presents OptABC, an innovative algorithm that enhances artificial bee colony optimization for hyperparameter tuning by integrating multiple techniques for better performance.

Findings

OptABC achieves faster convergence than traditional ABC.

OptABC outperforms existing hyperparameter tuning methods.

Experimental results confirm improved accuracy and efficiency.

Abstract

Hyperparameter tuning in machine learning algorithms is a computationally challenging task due to the large-scale nature of the problem. In order to develop an efficient strategy for hyper-parameter tuning, one promising solution is to use swarm intelligence algorithms. Artificial Bee Colony (ABC) optimization lends itself as a promising and efficient optimization algorithm for this purpose. However, in some cases, ABC can suffer from a slow convergence rate or execution time due to the poor initial population of solutions and expensive objective functions. To address these concerns, a novel algorithm, OptABC, is proposed to help ABC algorithm in faster convergence toward a near-optimum solution. OptABC integrates artificial bee colony algorithm, K-Means clustering, greedy algorithm, and opposition-based learning strategy for tuning the hyper-parameters of different machine learning models. OptABC employs these techniques in an attempt to diversify the initial population, and hence enhance the convergence ability without significantly decreasing the accuracy. In order to validate the performance of the proposed method, we compare the results with previous state-of-the-art approaches. Experimental results demonstrate the effectiveness of the OptABC compared to existing approaches in the literature.