Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging

TL;DR

This paper introduces OBIM, a novel method for merging large language models that reduces interference and improves performance by selectively preserving important parameters and incrementally merging models.

Contribution

OBIM is the first approach to mitigate both intra-model and inter-model interference during LLM merging through saliency-based selection and iterative merging framework.

Findings

OBIM outperforms existing merging methods in experiments.

It effectively reduces performance degradation caused by interference.

OBIM is applicable to both fine-tuned and pre-trained models.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities, but their high computational costs pose challenges for customization. Model merging offers a cost-effective alternative, yet existing methods suffer from interference among parameters, leading to performance degradation. In this work, we propose Optimal Brain Iterative Merging (OBIM), a novel method designed to mitigate both intra-model and inter-model interference. OBIM consists of two key components: (1) A saliency measurement mechanism that evaluates parameter importance based on loss changes induced by individual weight alterations, reducing intra-model interference by preserving only high-saliency parameters. (2) A mutually exclusive iterative merging framework, which incrementally integrates models using a binary mask to avoid direct parameter averaging, thereby mitigating inter-model interference. We validate OBIM through experiments on both Supervised Fine-Tuned (SFT) models and post-pretrained checkpoints. The results show that OBIM significantly outperforms existing merging techniques. Overall, OBIM provides an effective and practical solution for enhancing LLM merging.