Improving LoRA in Privacy-preserving Federated Learning

TL;DR

This paper introduces FFA-LoRA, an improved low-rank adaptation method for privacy-preserving federated learning, which stabilizes training, reduces communication costs, and enhances performance over traditional LoRA methods.

Contribution

The paper proposes FFA-LoRA, fixing certain matrices during training to address instability and hyper-parameter sensitivity in federated LoRA, improving efficiency and robustness.

Findings

FFA-LoRA achieves more stable training in FL scenarios.

It halves the communication cost compared to vanilla LoRA.

FFA-LoRA outperforms LoRA in various federated tasks.

Abstract

Low-rank adaptation (LoRA) is one of the most popular task-specific parameter-efficient fine-tuning (PEFT) methods on pre-trained language models for its good performance and computational efficiency. LoRA injects a product of two trainable rank decomposition matrices over the top of each frozen pre-trained model module. However, when applied in the setting of privacy-preserving federated learning (FL), LoRA may become unstable due to the following facts: 1) the effects of data heterogeneity and multi-step local updates are non-negligible, 2) additive noise enforced on updating gradients to guarantee differential privacy (DP) can be amplified and 3) the final performance is susceptible to hyper-parameters. A key factor leading to these phenomena is the discordance between jointly optimizing the two low-rank matrices by local clients and separately aggregating them by the central server. Thus, this paper proposes an efficient and effective version of LoRA, Federated Freeze A LoRA (FFA-LoRA), to alleviate these challenges and further halve the communication cost of federated fine-tuning LLMs. The core idea of FFA-LoRA is to fix the randomly initialized non-zero matrices and only fine-tune the zero-initialized matrices. Compared to LoRA, FFA-LoRA is motivated by practical and theoretical benefits in privacy-preserved FL. Our experiments demonstrate that FFA-LoRA provides more consistent performance with better computational efficiency over vanilla LoRA in various FL tasks.