Quantized and Asynchronous Federated Learning

TL;DR

This paper introduces QAFeL, a novel quantized asynchronous federated learning algorithm that improves scalability and communication efficiency while maintaining optimal convergence rates for non-convex objectives.

Contribution

We develop QAFeL, which integrates quantization into asynchronous federated learning with a hidden-state scheme and prove its optimal convergence without bounded gradients.

Findings

QAFeL achieves an $ ext{O}(1/ oot{T} ext{})$ convergence rate.

The algorithm handles unbounded gradients and variable client participation.

Experimental results validate theoretical convergence on benchmarks.

Abstract

Recent advances in federated learning have shown that asynchronous variants can be faster and more scalable than their synchronous counterparts. However, their design does not include quantization, which is necessary in practice to deal with the communication bottleneck. To bridge this gap, we develop a novel algorithm, Quantized Asynchronous Federated Learning (QAFeL), which introduces a hidden-state quantization scheme to avoid the error propagation caused by direct quantization. QAFeL also includes a buffer to aggregate client updates, ensuring scalability and compatibility with techniques such as secure aggregation. Furthermore, we prove that QAFeL achieves an $\mathcal{O}(1/\sqrt{T})$ ergodic convergence rate for stochastic gradient descent on non-convex objectives, which is the optimal order of complexity, without requiring bounded gradients or uniform client arrivals. We also prove that the cross-term error between staleness and quantization only affects the higher-order error terms. We validate our theoretical findings on standard benchmarks.