Fed-DLoRA: Efficient Wireless Federated Learning with Dynamic Low-Rank Adaptation

TL;DR

Fed-DLoRA introduces a dynamic low-rank adaptation method for wireless federated learning, significantly reducing communication costs and improving training efficiency in vehicular networks.

Contribution

The paper proposes Fed-DLoRA, a novel FL algorithm combining low-rank adaptation with an adaptive resource scheduling strategy, enhancing performance in dynamic wireless environments.

Findings

Fed-DLoRA outperforms traditional FL methods in accuracy and convergence speed.

The adaptive rank and resource scheduling algorithm improves communication efficiency.

Theoretical analysis links LoRA rank, scheduling, and convergence.

Abstract

Federated learning (FL) offers a promising distributed learning paradigm for internet of vehicles (IoV) applications. However, it faces challenges from communication overhead and dynamic environments. Model compression techniques reduce computing and communication burden yet create trade-offs between compression ratios and vehicle participation strategies. In this paper, we propose a lightweight FL algorithm named federated learning with dynamic low-rank adaptation (Fed-DLoRA), which is combined with low-rank adaptation (LoRA) to effectively reduce parameters and communication costs while enhancing training efficiency. The convergence analysis of Fed-DLoRA is conducted through stochastic gradient descent optimization coupled with singular value decomposition. This analysis establishes the theoretical relationships among LoRA rank, vehicular scheduling strategies and the model's convergence characteristics. Building on these insights, we formulate a joint optimization problem aimed at maximizing system performance. To address this problem, we propose an adaptive rank, bandwidth and vehicle selection (ARBVS) algorithm that integrates enumeration with greedy optimization strategies. The algorithm provides efficient rank selection and resource scheduling strategies for each FL communication round, thereby achieving effective performance improvements for the FL system. Experimental results demonstrate that Fed-DLoRA achieves superior performance compared to conventional federated learning approaches, exhibiting enhanced accuracy, faster convergence, and improved communication efficiency.