Model merging with SVD to tie the Knots

TL;DR

This paper introduces KnOTS, a method using SVD to align LoRA finetuned models for improved merging, addressing the lower alignment issue and enhancing performance across vision and language tasks.

Contribution

We propose KnOTS, a novel SVD-based approach to improve the merging of LoRA finetuned models by enhancing their alignment, and introduce a new benchmark for evaluating merged model generality.

Findings

KnOTS improves LoRA merging performance by up to 4.3%.

The method enhances model alignment, leading to better task generalization.

A new benchmark evaluates the generality of merged models.

Abstract

Recent model merging methods demonstrate that the parameters of fully-finetuned models specializing in distinct tasks can be combined into one model capable of solving all tasks without retraining. Yet, this success does not transfer well when merging LoRA finetuned models. We study this phenomenon and observe that the weights of LoRA finetuned models showcase a lower degree of alignment compared to their fully-finetuned counterparts. We hypothesize that improving this alignment is key to obtaining better LoRA model merges, and propose KnOTS to address this problem. KnOTS uses the SVD to jointly transform the weights of different LoRA models into an aligned space, where existing merging methods can be applied. In addition, we introduce a new benchmark that explicitly evaluates whether merged models are general models. Notably, KnOTS consistently improves LoRA merging by up to 4.3% across several vision and language benchmarks, including our new setting. We release our code at: https://github.com/gstoica27/KnOTS.