Personalized Wireless Federated Learning for Large Language Models

TL;DR

This paper introduces a personalized federated learning framework for large language models in wireless networks, addressing privacy, resource, and communication challenges through innovative adaptation and optimization techniques.

Contribution

It proposes the PWFF framework with adapter, LoRA, personalized loss, and multi-objective alignment to improve efficiency and personalization in wireless federated LLM training.

Findings

Reduced energy consumption via LoRA and adapters.

Lower communication delay with global partial aggregation.

Enhanced personalization and stability in federated LLM training.

Abstract

Large language models (LLMs) have driven profound transformations in wireless networks. However, within wireless environments, the training of LLMs faces significant challenges related to security and privacy. Federated Learning (FL), with its decentralized architecture, offers enhanced data privacy protection. Nevertheless, when integrated with LLMs, FL still struggles with several critical limitations, including large-scale and heterogeneous data, resource-intensive training, and substantial communication overhead. To address these challenges, this paper first presents a systematic analysis of the distinct training stages of LLMs in wireless networks, including pre-training, instruction tuning, and alignment tuning. Building upon this foundation, we propose a Personalized Wireless Federated Fine-tuning (PWFF) framework. Initially, we utilize the adapter and Low-Rank Adaptation (LoRA) techniques to decrease energy consumption, while employing global partial aggregation to reduce communication delay. Subsequently, we develop two reward models and design a personalized loss function to fulfill the goal of personalized learning. Furthermore, we implement a local multi-objective alignment to ensure the stability and effectiveness of the FL process. Finally, we conduct a series of simulations to validate the performance of the proposed PWFF method and provide an in-depth discussion of the open issues.