DPI: Exploiting Parameter Heterogeneity for Interference-Free Fine-Tuning

TL;DR

This paper introduces DPI, a method that isolates task-specific parameters during fine-tuning of large language models to prevent interference and improve performance across multiple tasks.

Contribution

DPI proposes a novel parameter isolation approach that identifies and freezes core parameters per task, reducing cross-task interference during fine-tuning.

Findings

Consistently reduces data conflicts in multi-task fine-tuning

Achieves performance improvements over baseline methods

Effectively isolates task-specific parameter regions

Abstract

Supervised fine-tuning (SFT) is a crucial step for adapting large language models (LLMs) to downstream tasks. However, conflicting objectives across heterogeneous SFT tasks often induce the "seesaw effect": optimizing for one task may degrade performance on others, particularly when model parameters are updated indiscriminately. In this paper, we propose a principled approach to disentangle and isolate task-specific parameter regions, motivated by the hypothesis that parameter heterogeneity underlies cross-task interference. Specifically, we first independently fine-tune LLMs on diverse SFT tasks and identify each task's core parameter region as the subset of parameters exhibiting the largest updates. Tasks with highly overlapping core parameter regions are merged for joint training, while disjoint tasks are organized into different stages. During multi-stage SFT, core parameters acquired in prior tasks are frozen, thereby preventing overwriting by subsequent tasks. To verify the effectiveness of our method, we conducted intensive experiments on multiple public datasets. The results showed that our dynamic parameter isolation strategy consistently reduced data conflicts and achieved consistent performance improvements compared to multi-stage and multi-task tuning baselines.