Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks

TL;DR

This paper introduces FedQVR, a communication-efficient federated learning algorithm that reduces variance and handles device heterogeneity with quantized transmissions, improving convergence and reducing communication costs in wireless edge networks.

Contribution

The paper proposes FedQVR, a novel variance-reduction federated learning algorithm robust to heterogeneity and quantization, with an extension FedQVR-E for wireless channel constraints.

Findings

FedQVR achieves significant communication savings while maintaining convergence.

FedQVR is robust to device heterogeneity and quantization levels.

FedQVR-E improves convergence under wireless channel constraints.

Abstract

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.