FedDQ: Communication-Efficient Federated Learning with Descending Quantization

TL;DR

FedDQ introduces a descending quantization scheme for federated learning that reduces communication costs by adapting the quantization level downward as training progresses, based on theoretical insights.

Contribution

The paper proposes FedDQ, a novel descending quantization method for FL that improves communication efficiency by aligning quantization levels with model update ranges.

Findings

Reduces communication volume by up to 65.2%.

Decreases communication rounds by up to 68%.

Theoretically justified the decreasing quantization level approach.

Abstract

Federated learning (FL) is an emerging learning paradigm without violating users' privacy. However, large model size and frequent model aggregation cause serious communication bottleneck for FL. To reduce the communication volume, techniques such as model compression and quantization have been proposed. Besides the fixed-bit quantization, existing adaptive quantization schemes use ascending-trend quantization, where the quantization level increases with the training stages. In this paper, we first investigate the impact of quantization on model convergence, and show that the optimal quantization level is directly related to the range of the model updates. Given the model is supposed to converge with the progress of the training, the range of the model updates will gradually shrink, indicating that the quantization level should decrease with the training stages. Based on the theoretical analysis, a descending quantization scheme named FedDQ is proposed. Experimental results show that the proposed descending quantization scheme can save up to 65.2% of the communicated bit volume and up to 68% of the communication rounds, when compared with existing schemes.