A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning

TL;DR

This paper introduces FedLuck, an adaptive framework for asynchronous federated learning that jointly optimizes local update frequency and gradient compression, significantly reducing communication and training time in heterogeneous, low-bandwidth environments.

Contribution

It proposes a novel joint optimization approach for local updates and gradient compression in AFL, with a theoretical analysis and an adaptive framework called FedLuck.

Findings

Reduces communication by 56%

Speeds up training by 55%

Maintains competitive accuracy in heterogeneous settings

Abstract

Asynchronous Federated Learning (AFL) confronts inherent challenges arising from the heterogeneity of devices (e.g., their computation capacities) and low-bandwidth environments, both potentially causing stale model updates (e.g., local gradients) for global aggregation. Traditional approaches mitigating the staleness of updates typically focus on either adjusting the local updating or gradient compression, but not both. Recognizing this gap, we introduce a novel approach that synergizes local updating with gradient compression. Our research begins by examining the interplay between local updating frequency and gradient compression rate, and their collective impact on convergence speed. The theoretical upper bound shows that the local updating frequency and gradient compression rate of each device are jointly determined by its computing power, communication capabilities and other factors. Building on this foundation, we propose an AFL framework called FedLuck that adaptively optimizes both local update frequency and gradient compression rates. Experiments on image classification and speech recognization show that FedLuck reduces communication consumption by 56% and training time by 55% on average, achieving competitive performance in heterogeneous and low-bandwidth scenarios compared to the baselines.