It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization

TL;DR

This paper presents MM-MO, a multi-objective optimization approach that automates large language model merging, improving performance across tasks without human intervention by leveraging advanced optimization techniques.

Contribution

The paper introduces MM-MO, a novel automated multi-objective optimization method for model merging that reduces reliance on human knowledge and enhances merging quality across multiple tasks.

Findings

MM-MO outperforms existing merging methods in experiments.

Automated optimization reduces human effort in model merging.

Incorporating sparsity improves generalization across tasks.

Abstract

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, model merging faces two significant challenges: First, existing methods rely heavily on human knowledge or intuition. Second, it's difficult to obtain the great model merging configuration in limited evaluations. To address these challenges, we formalize model merging as a multi-objective optimization problem and propose an automated optimization approach named MM-MO. This method leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention. In MM-MO, a weak-to-strong method is employed to enhance the acquisition function, allowing previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is applied to screen these configurations, increasing the possibility of identifying high-quality merging configuration. Additionally, we designed a sparsity metric as an additional optimization objective to enhance the model's generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MM-MO algorithm is competitive and effective in achieving high-quality model merging.