Leveraging Channel Noise for Sampling and Privacy via Quantized Federated Langevin Monte Carlo

TL;DR

This paper introduces quantized federated Langevin Monte Carlo (FLMC), which uses channel noise and gradient quantization to improve Bayesian sampling and privacy in wireless federated learning systems.

Contribution

It proposes a novel FLMC method combining one-bit gradient quantization with channel-driven sampling, enhancing privacy and efficiency over prior analog approaches.

Findings

Digital implementation outperforms analog at high SNR for privacy.

Channel noise can be exploited for Monte Carlo sampling.

Quantization reduces communication costs while maintaining sampling quality.

Abstract

For engineering applications of artificial intelligence, Bayesian learning holds significant advantages over standard frequentist learning, including the capacity to quantify uncertainty. Langevin Monte Carlo (LMC) is an efficient gradient-based approximate Bayesian learning strategy that aims at producing samples drawn from the posterior distribution of the model parameters. Prior work focused on a distributed implementation of LMC over a multi-access wireless channel via analog modulation. In contrast, this paper proposes quantized federated LMC (FLMC), which integrates one-bit stochastic quantization of the local gradients with channel-driven sampling. Channel-driven sampling leverages channel noise for the purpose of contributing to Monte Carlo sampling, while also serving the role of privacy mechanism. Analog and digital implementations of wireless LMC are compared as a function of differential privacy (DP) requirements, revealing the advantages of the latter at sufficiently high signal-to-noise ratio.