MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm

TL;DR

This paper presents MeGA, a genetic algorithm-based method for merging multiple pre-trained neural networks, which enhances model accuracy and robustness by optimally combining weights.

Contribution

It introduces a novel genetic algorithm approach for effectively merging independently trained neural networks, outperforming traditional averaging and ensemble techniques.

Findings

Improved test accuracy on CIFAR-10 compared to individual models.

Enhanced robustness of merged models.

Scalable method for integrating multiple pre-trained networks.

Abstract

In this paper, we introduce a novel method for merging the weights of multiple pre-trained neural networks using a genetic algorithm called MeGA. Traditional techniques, such as weight averaging and ensemble methods, often fail to fully harness the capabilities of pre-trained networks. Our approach leverages a genetic algorithm with tournament selection, crossover, and mutation to optimize weight combinations, creating a more effective fusion. This technique allows the merged model to inherit advantageous features from both parent models, resulting in enhanced accuracy and robustness. Through experiments on the CIFAR-10 dataset, we demonstrate that our genetic algorithm-based weight merging method improves test accuracy compared to individual models and conventional methods. This approach provides a scalable solution for integrating multiple pre-trained networks across various deep learning applications. Github is available at: https://github.com/YUNBLAK/MeGA-Merging-Multiple-Independently-Trained-Neural-Networks-Based-on-Genetic-Algorithm