LoLaFL: Low-Latency Federated Learning via Forward-only Propagation

TL;DR

LoLaFL introduces a low-latency federated learning framework using forward-only propagation and layer-wise communication, significantly reducing latency while maintaining accuracy, suitable for 6G mobile networks.

Contribution

The paper proposes LoLaFL, a novel federated learning framework that employs forward-only propagation and nonlinear aggregation schemes to drastically cut communication rounds and latency.

Findings

Achieves over 87% latency reduction with comparable accuracy.

Employs layer-wise transmission and aggregation for efficiency.

Introduces nonlinear aggregation schemes based on harmonic mean and low-rank features.

Abstract

Federated learning (FL) has emerged as a widely adopted paradigm for enabling edge learning with distributed data while ensuring data privacy. However, the traditional FL with deep neural networks trained via backpropagation can hardly meet the low-latency learning requirements in the sixth generation (6G) mobile networks. This challenge mainly arises from the high-dimensional model parameters to be transmitted and the numerous rounds of communication required for convergence due to the inherent randomness of the training process. To address this issue, we adopt the state-of-the-art principle of maximal coding rate reduction to learn linear discriminative features and extend the resultant white-box neural network into FL, yielding the novel framework of Low-Latency Federated Learning (LoLaFL) via forward-only propagation. LoLaFL enables layer-wise transmissions and aggregation with significantly fewer communication rounds, thereby considerably reducing latency. Additionally, we propose two \emph{nonlinear} aggregation schemes for LoLaFL. The first scheme is based on the proof that the optimal NN parameter aggregation in LoLaFL should be harmonic-mean-like. The second scheme further exploits the low-rank structures of the features and transmits the low-rank-approximated covariance matrices of features to achieve additional latency reduction. Theoretic analysis and experiments are conducted to evaluate the performance of LoLaFL. In comparison with traditional FL, the two nonlinear aggregation schemes for LoLaFL can achieve reductions in latency of over 87\% and 97\%, respectively, while maintaining comparable accuracies.