LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint

TL;DR

LED-Merging is a novel, training-free framework that effectively mitigates safety-utility conflicts in model merging by locating, selecting, and isolating task-specific neurons, leading to safer and more reliable multi-task LLMs.

Contribution

It introduces a three-stage neuron-level merging framework that addresses neuron misidentification and interference, improving safety and utility in model merging without additional training.

Findings

Reduces harmful response rates by 31.4% on HarmBench.

Preserves 95% of utility performance, e.g., 52.39% accuracy on GSM8K.

Effective across multiple LLM architectures like Llama-3-8B, Mistral-7B, and Llama2-13B.

Abstract

Fine-tuning pre-trained Large Language Models (LLMs) for specialized tasks incurs substantial computational and data costs. While model merging offers a training-free solution to integrate multiple task-specific models, existing methods suffer from safety-utility conflicts where enhanced general capabilities degrade safety safeguards. We identify two root causes: $\textbf{neuron misidentification}$ due to simplistic parameter magnitude-based selection, and $\textbf{cross-task neuron interference}$ during merging. To address these challenges, we propose $\textbf{LED-Merging}$, a three-stage framework that $\textbf{L}$ocates task-specific neurons via gradient-based attribution, dynamically $\textbf{E}$lects critical neurons through multi-model importance fusion, and $\textbf{D}$isjoints conflicting updates through parameter isolation. Extensive experiments on Llama-3-8B, Mistral-7B, and Llama2-13B demonstrate that LED-Merging effectively reduces harmful response rates, showing a 31.4\% decrease on Llama-3-8B-Instruct on HarmBench, while simultaneously preserving 95\% of utility performance, such as achieving 52.39\% accuracy on GSM8K. LED-Merging resolves safety-utility conflicts and provides a lightweight, training-free paradigm for constructing reliable multi-task LLMs. Code is available at $\href{https://github.com/MqLeet/LED-Merging}{GitHub}$.