Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models

TL;DR

This paper introduces FedTT and FedTT+ methods that enable communication-efficient, privacy-preserving federated fine-tuning of large language models using tensorized adapters, effectively handling data heterogeneity.

Contribution

The paper proposes novel tensorized adapter-based federated fine-tuning methods, FedTT and FedTT+, that reduce communication costs and improve robustness against data heterogeneity in federated learning.

Findings

Achieved up to 10× reduction in communication cost.

Performed on par or better than existing federated PEFT methods.

Demonstrated effectiveness on BERT and LLaMA models.

Abstract

Parameter-efficient fine-tuning (PEFT) methods typically assume that Large Language Models (LLMs) are trained on data from a single device or client. However, real-world scenarios often require fine-tuning these models on private data distributed across multiple devices. Federated Learning (FL) offers an appealing solution by preserving user privacy, as sensitive data remains on local devices during training. Nonetheless, integrating PEFT methods into FL introduces two main challenges: communication overhead and data heterogeneity. In this paper, we introduce FedTT and FedTT+, methods for adapting LLMs by integrating tensorized adapters into client-side models' encoder/decoder blocks. FedTT is versatile and can be applied to both cross-silo FL and large-scale cross-device FL. FedTT+, an extension of FedTT tailored for cross-silo FL, enhances robustness against data heterogeneity by adaptively freezing portions of tensor factors, further reducing the number of trainable parameters. Experiments on BERT and LLaMA models demonstrate that our proposed methods successfully address data heterogeneity challenges and perform on par or even better than existing federated PEFT approaches while achieving up to 10$\times$ reduction in communication cost.