Parameter Efficient Multi-task Model Fusion with Partial Linearization

TL;DR

This paper introduces a partial linearization method for multi-task model fusion that enhances the efficiency and effectiveness of combining fine-tuned large pre-trained models for multiple tasks.

Contribution

It proposes a novel partial linearization technique for adapter modules that improves multi-task fusion in parameter-efficient fine-tuning methods like LoRA.

Findings

Outperforms standard adapter tuning in multi-task settings

Enables more effective fusion of multiple fine-tuned task vectors

Demonstrates scalability with increasing number of tasks

Abstract

Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven to be powerful and scalable to incorporate fine-tuned weights from different tasks into a multi-task model. However, efficiently fine-tuning large pre-trained models on multiple downstream tasks remains challenging, leading to inefficient multi-task model fusion. In this work, we propose a novel method to improve multi-task fusion for parameter-efficient fine-tuning techniques like LoRA fine-tuning. Specifically, our approach partially linearizes only the adapter modules and applies task arithmetic over the linearized adapters. This allows us to leverage the the advantages of model fusion over linearized fine-tuning, while still performing fine-tuning and inference efficiently. We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model, outperforming standard adapter tuning and task arithmetic alone. Experimental results demonstrate the capabilities of our proposed partial linearization technique to effectively construct unified multi-task models via the fusion of fine-tuned task vectors. We evaluate performance over an increasing number of tasks and find that our approach outperforms standard parameter-efficient fine-tuning techniques. The results highlight the benefits of partial linearization for scalable and efficient multi-task model fusion. The code is available at https://github.com/tanganke/peta