Extended Mixture of MLP Experts by Hybrid of Conjugate Gradient Method and Modified Cuckoo Search

TL;DR

This paper introduces a hybrid optimization approach combining Modified Cuckoo Search and Conjugate Gradient to enhance the learning efficiency and accuracy of Mixture of Experts models, outperforming traditional methods.

Contribution

It proposes a novel hybrid optimization method for ME models that integrates MCS and CG, improving convergence speed and performance in classification and regression tasks.

Findings

Faster convergence compared to traditional methods

Improved accuracy in benchmark datasets

Superior performance in classification and regression

Abstract

This paper investigates a new method for improving the learning algorithm of Mixture of Experts (ME) model using a hybrid of Modified Cuckoo Search (MCS) and Conjugate Gradient (CG) as a second order optimization technique. The CG technique is combined with Back-Propagation (BP) algorithm to yield a much more efficient learning algorithm for ME structure. In addition, the experts and gating networks in enhanced model are replaced by CG based Multi-Layer Perceptrons (MLPs) to provide faster and more accurate learning. The CG is considerably depends on initial weights of connections of Artificial Neural Network (ANN), so, a metaheuristic algorithm, the so-called Modified Cuckoo Search is applied in order to select the optimal weights. The performance of proposed method is compared with Gradient Decent Based ME (GDME) and Conjugate Gradient Based ME (CGME) in classification and regression problems. The experimental results show that hybrid MSC and CG based ME (MCS-CGME) has faster convergence and better performance in utilized benchmark data sets.