Joint Privacy Enhancement and Quantization in Federated Learning

TL;DR

This paper introduces JoPEQ, a method that combines lossy compression and privacy preservation in federated learning, achieving efficient model updates with strong privacy guarantees and resistance to attacks.

Contribution

JoPEQ is the first approach to jointly implement privacy enhancement and quantization in federated learning using vector quantization with lattice-based noise.

Findings

JoPEQ maintains model utility while achieving specified privacy levels.

JoPEQ provides analytical guarantees for privacy, distortion, and convergence.

Numerical studies confirm JoPEQ's robustness against privacy attacks.

Abstract

Federated learning (FL) is an emerging paradigm for training machine learning models using possibly private data available at edge devices. The distributed operation of FL gives rise to challenges that are not encountered in centralized machine learning, including the need to preserve the privacy of the local datasets, and the communication load due to the repeated exchange of updated models. These challenges are often tackled individually via techniques that induce some distortion on the updated models, e.g., local differential privacy (LDP) mechanisms and lossy compression. In this work we propose a method coined joint privacy enhancement and quantization (JoPEQ), which jointly implements lossy compression and privacy enhancement in FL settings. In particular, JoPEQ utilizes vector quantization based on random lattice, a universal compression technique whose byproduct distortion is statistically equivalent to additive noise. This distortion is leveraged to enhance privacy by augmenting the model updates with dedicated multivariate privacy preserving noise. We show that JoPEQ simultaneously quantizes data according to a required bit-rate while holding a desired privacy level, without notably affecting the utility of the learned model. This is shown via analytical LDP guarantees, distortion and convergence bounds derivation, and numerical studies. Finally, we empirically assert that JoPEQ demolishes common attacks known to exploit privacy leakage.