$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning

TL;DR

Trans-LoRA enables nearly data-free transfer of LoRA modules across different models using synthetic data, addressing privacy and proprietary data constraints in model fine-tuning.

Contribution

It introduces a novel synthetic data-based method for lossless transfer of LoRA modules across models without requiring original training data.

Findings

Effective transfer of LoRA modules within and across model families.

Achieves lossless or improved performance in transfer tasks.

Works across different PEFT methods and diverse tasks.

Abstract

Low-rank adapters (LoRA) and their variants are popular parameter-efficient fine-tuning (PEFT) techniques that closely match full model fine-tune performance while requiring only a small number of additional parameters. These additional LoRA parameters are specific to the base model being adapted. When the base model needs to be deprecated and replaced with a new one, all the associated LoRA modules need to be re-trained. Such re-training requires access to the data used to train the LoRA for the original base model. This is especially problematic for commercial cloud applications where the LoRA modules and the base models are hosted by service providers who may not be allowed to host proprietary client task data. To address this challenge, we propose $\textit{Trans-LoRA}$ -- a novel method for lossless, nearly data-free transfer of LoRAs across base models. Our approach relies on synthetic data to transfer LoRA modules. Using large language models, we design a synthetic data generator to approximate the data-generating process of the $\textit{observed}$ task data subset. Training on the resulting synthetic dataset transfers LoRA modules to new models. We show the effectiveness of our approach using both LLama and Gemma model families. Our approach achieves lossless (mostly improved) LoRA transfer between models within and across different base model families, and even between different PEFT methods, on a wide variety of tasks.