The Effect of Quantization in Federated Learning: A R\'enyi Differential Privacy Perspective

TL;DR

This paper explores how quantization affects privacy in federated learning by analyzing the privacy guarantees of quantized Gaussian mechanisms using RDP, showing that lower quantization bits can improve privacy protection.

Contribution

It provides a theoretical analysis of the impact of quantization on privacy in FL using RDP and validates findings with membership inference attacks.

Findings

Lower quantization bits improve privacy protection.

Numerical results confirm theoretical privacy guarantees.

Quantization can reduce communication overhead while enhancing privacy.

Abstract

Federated Learning (FL) is an emerging paradigm that holds great promise for privacy-preserving machine learning using distributed data. To enhance privacy, FL can be combined with Differential Privacy (DP), which involves adding Gaussian noise to the model weights. However, FL faces a significant challenge in terms of large communication overhead when transmitting these model weights. To address this issue, quantization is commonly employed. Nevertheless, the presence of quantized Gaussian noise introduces complexities in understanding privacy protection. This research paper investigates the impact of quantization on privacy in FL systems. We examine the privacy guarantees of quantized Gaussian mechanisms using R\'enyi Differential Privacy (RDP). By deriving the privacy budget of quantized Gaussian mechanisms, we demonstrate that lower quantization bit levels provide improved privacy protection. To validate our theoretical findings, we employ Membership Inference Attacks (MIA), which gauge the accuracy of privacy leakage. The numerical results align with our theoretical analysis, confirming that quantization can indeed enhance privacy protection. This study not only enhances our understanding of the correlation between privacy and communication in FL but also underscores the advantages of quantization in preserving privacy.