FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devices

TL;DR

FedPT is a federated proxy-tuning framework that enables privacy-preserving, resource-efficient fine-tuning of large language models on edge devices by only requiring output predictions, not model parameters.

Contribution

This paper introduces FedPT, a novel approach that allows federated fine-tuning of large LMs using only output predictions, reducing resource overhead and preserving privacy.

Findings

FedPT achieves performance comparable to direct fine-tuning of large LMs.

Significantly reduces computation, communication, and memory costs.

Enables large LM fine-tuning on resource-constrained edge devices.

Abstract

Despite demonstrating superior performance across a variety of linguistic tasks, pre-trained large language models (LMs) often require fine-tuning on specific datasets to effectively address different downstream tasks. However, fine-tuning these LMs for downstream tasks necessitates collecting data from individuals, which raises significant privacy concerns. Federated learning (FL) has emerged as the de facto solution, enabling collaborative model training without sharing raw data. While promising, federated fine-tuning of large LMs faces significant challenges, including restricted access to model parameters and high computation, communication, and memory overhead. To address these challenges, this paper introduces \textbf{Fed}erated \textbf{P}roxy-\textbf{T}uning (FedPT), a novel framework for federated fine-tuning of black-box large LMs, requiring access only to their predictions over the output vocabulary instead of their parameters. Specifically, devices in FedPT first collaboratively tune a smaller LM, and then the server combines the knowledge learned by the tuned small LM with the knowledge learned by the larger pre-trained LM to construct a large proxy-tuned LM that can reach the performance of directly tuned large LMs. The experimental results demonstrate that FedPT can significantly reduce computation, communication, and memory overhead while maintaining competitive performance compared to directly federated fine-tuning of large LMs. FedPT offers a promising solution for efficient, privacy-preserving fine-tuning of large LMs on resource-constrained devices, broadening the accessibility and applicability of state-of-the-art large LMs.