PSO-Merging: Merging Models Based on Particle Swarm Optimization

TL;DR

This paper introduces PSO-Merging, a data-driven particle swarm optimization method for efficiently merging multiple language models, outperforming existing techniques in scalability and effectiveness.

Contribution

The paper proposes a novel PSO-based model merging approach that overcomes computational challenges of gradient-based methods and improves performance over existing gradient-free techniques.

Findings

PSO-Merging outperforms baseline methods in language model merging tasks.

The method is more scalable and computationally efficient.

Experimental results demonstrate improved model performance.

Abstract

Model merging has emerged as an efficient strategy for constructing multitask models by integrating the strengths of multiple available expert models, thereby reducing the need to fine-tune a pre-trained model for all the tasks from scratch. Existing data-independent methods struggle with performance limitations due to the lack of data-driven guidance. Data-driven approaches also face key challenges: gradient-based methods are computationally expensive, limiting their practicality for merging large expert models, whereas existing gradient-free methods often fail to achieve satisfactory results within a limited number of optimization steps. To address these limitations, this paper introduces PSO-Merging, a novel data-driven merging method based on the Particle Swarm Optimization (PSO). In this approach, we initialize the particle swarm with a pre-trained model, expert models, and sparsified expert models. We then perform multiple iterations, with the final global best particle serving as the merged model. Experimental results on different language models show that PSO-Merging generally outperforms baseline merging methods, offering a more efficient and scalable solution for model merging.