Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models

TL;DR

Ferret introduces a scalable federated full-parameter tuning method for large language models that balances communication efficiency and model accuracy through shared randomness and low-dimensional projections.

Contribution

It is the first first-order federated tuning method for LLMs that maintains accuracy while reducing communication overhead using shared randomness and low-dimensional updates.

Findings

Achieves high computational efficiency in federated tuning.

Reduces communication overhead significantly.

Maintains competitive model accuracy.

Abstract

Large Language Models (LLMs) have become indispensable in numerous real-world applications. However, fine-tuning these models at scale, especially in federated settings where data privacy and communication efficiency are critical, presents significant challenges. Existing approaches often resort to parameter-efficient fine-tuning (PEFT) to mitigate communication overhead, but this typically comes at the cost of model accuracy. To this end, we propose federated full-parameter tuning at scale for LLMs (Ferret), the first first-order method with shared randomness to enable scalable full-parameter tuning of LLMs across decentralized data sources while maintaining competitive model accuracy. Ferret accomplishes this through three aspects: (i) it employs widely used first-order methods for efficient local updates; (ii) it projects these updates into a low-dimensional space to considerably reduce communication overhead; and (iii) it reconstructs local updates from this low-dimensional space with shared randomness to facilitate effective full-parameter global aggregation, ensuring fast convergence and competitive final performance. Our rigorous theoretical analyses and insights along with extensive experiments, show that Ferret significantly enhances the scalability of existing federated full-parameter tuning approaches by achieving high computational efficiency, reduced communication overhead, and fast convergence, all while maintaining competitive model accuracy. Our implementation is available at https://github.com/allen4747/Ferret.