Communication-Efficient and Privacy-Adaptable Mechanism for Federated Learning

TL;DR

This paper introduces CEPAM, a communication-efficient and privacy-adaptable mechanism for federated learning that combines randomized vector quantization with differential privacy, improving accuracy and privacy trade-offs.

Contribution

The paper proposes CEPAM, a novel federated learning mechanism that achieves simultaneous communication efficiency and privacy adaptability using rejection-sampled universal quantization.

Findings

CEPAM provides strong privacy guarantees with customizable protection levels.

CEPAM outperforms baseline models in learning accuracy on MNIST.

Theoretical analysis confirms privacy and utility trade-offs.

Abstract

Training machine learning models on decentralized private data via federated learning (FL) poses two key challenges: communication efficiency and privacy protection. In this work, we address these challenges within the trusted aggregator model by introducing a novel approach called the Communication-Efficient and Privacy-Adaptable Mechanism (CEPAM), achieving both objectives simultaneously. In particular, CEPAM leverages the rejection-sampled universal quantizer (RSUQ), a construction of randomized vector quantizer whose resulting distortion is equivalent to a prescribed noise, such as Gaussian or Laplace noise, enabling joint differential privacy and compression. Our CEPAM provides the additional benefit of privacy adaptability, allowing clients and the server to customize privacy protection based on required accuracy and protection. We theoretically analyze the privacy guarantee of CEPAM and investigate the trade-offs among user privacy and accuracy of CEPAM through experimental evaluations. Moreover, we assess CEPAM's utility performance using MNIST dataset, demonstrating that CEPAM surpasses baseline models in terms of learning accuracy.