FroM: Frobenius Norm-Based Data-Free Adaptive Model Merging

TL;DR

This paper introduces FroM, a data-free adaptive model merging method based on Frobenius norm, which improves knowledge fusion from multiple models without training data, reducing task interference in fine-tuning scenarios.

Contribution

It proposes FroM, a novel Frobenius norm-based model merging technique that directly measures model parameters and outperforms existing methods without requiring training data.

Findings

FroM outperforms baseline methods in various fine-tuning scenarios.

FroM effectively alleviates task interference during model merging.

The method requires no training data and introduces an adjustable hyperparameter.

Abstract

With the development of large language models, fine-tuning has emerged as an effective method to enhance performance in specific scenarios by injecting domain-specific knowledge. In this context, model merging techniques provide a solution for fusing knowledge from multiple fine-tuning models by combining their parameters. However, traditional methods often encounter task interference when merging full fine-tuning models, and this problem becomes even more evident in parameter-efficient fine-tuning scenarios. In this paper, we introduce an improvement to the RegMean method, which indirectly leverages the training data to approximate the outputs of the linear layers before and after merging. We propose an adaptive merging method called FroM, which directly measures the model parameters using the Frobenius norm, without any training data. By introducing an additional hyperparameter for control, FroM outperforms baseline methods across various fine-tuning scenarios, alleviating the task interference problem.