Privacy-Preserving Quantized Federated Learning with Diverse Precision

TL;DR

This paper proposes a novel federated learning approach that preserves privacy and handles diverse quantization resolutions across devices, improving utility and security in distributed machine learning.

Contribution

It introduces a stochastic quantizer that ensures differential privacy with minimal error and a cluster size optimization method for better model aggregation.

Findings

Enhanced privacy protection with bounded distortion.

Improved learning utility over traditional methods.

Effective handling of heterogeneity in device quantization resolutions.

Abstract

Federated learning (FL) has emerged as a promising paradigm for distributed machine learning, enabling collaborative training of a global model across multiple local devices without requiring them to share raw data. Despite its advancements, FL is limited by factors such as: (i) privacy risks arising from the unprotected transmission of local model updates to the fusion center (FC) and (ii) decreased learning utility caused by heterogeneity in model quantization resolution across participating devices. Prior work typically addresses only one of these challenges because maintaining learning utility under both privacy risks and quantization heterogeneity is a non-trivial task. In this paper, our aim is therefore to improve the learning utility of a privacy-preserving FL that allows clusters of devices with different quantization resolutions to participate in each FL round. Specifically, we introduce a novel stochastic quantizer (SQ) that is designed to simultaneously achieve differential privacy (DP) and minimum quantization error. Notably, the proposed SQ guarantees bounded distortion, unlike other DP approaches. To address quantization heterogeneity, we introduce a cluster size optimization technique combined with a linear fusion approach to enhance model aggregation accuracy. Numerical simulations validate the benefits of our approach in terms of privacy protection and learning utility compared to the conventional LaplaceSQ-FL algorithm.