Communication-Efficient Federated Learning via Clipped Uniform Quantization

TL;DR

This paper introduces a clipped uniform quantization method for federated learning that reduces communication costs while maintaining high accuracy, using optimal clipping and adaptive schemes to improve robustness and privacy.

Contribution

The paper proposes a novel clipped uniform quantization approach with adaptive thresholds, enhancing communication efficiency and privacy in federated learning without sacrificing model performance.

Findings

Achieves near-full-precision accuracy with fewer bits.

Reduces bandwidth and memory requirements significantly.

Enhances privacy by not requiring client data volume disclosure.

Abstract

This paper presents a novel approach to enhance communication efficiency in federated learning through clipped uniform quantization. By leveraging optimal clipping thresholds and client-specific adaptive quantization schemes, the proposed method significantly reduces bandwidth and memory requirements for model weight transmission between clients and the server while maintaining competitive accuracy. We investigate the effects of symmetric clipping and uniform quantization on model performance, emphasizing the role of stochastic quantization in mitigating artifacts and improving robustness. Extensive simulations demonstrate that the method achieves near-full-precision performance with substantial communication savings. Moreover, the proposed approach facilitates efficient weight averaging based on the inverse of the mean squared quantization errors, effectively balancing the trade-off between communication efficiency and model accuracy. Moreover, in contrast to federated averaging, this design obviates the need to disclose client-specific data volumes to the server, thereby enhancing client privacy. Comparative analysis with conventional quantization methods further confirms the efficacy of the proposed scheme.