The Poisson binomial mechanism for secure and private federated learning

TL;DR

This paper introduces the Poisson Binomial mechanism (PBM), a discrete differential privacy method for federated learning that offers improved privacy-accuracy trade-offs, lower communication costs, and compatibility with secure aggregation protocols.

Contribution

The paper presents the PBM, a novel discrete DP mechanism with bounded support, enabling efficient privacy-preserving federated learning without modular clipping and with improved communication efficiency.

Findings

Achieves privacy-accuracy trade-offs comparable to Gaussian mechanisms.

Supports secure aggregation without modular clipping.

Reduces communication cost as privacy constraints tighten.

Abstract

We introduce the Poisson Binomial mechanism (PBM), a discrete differential privacy mechanism for distributed mean estimation (DME) with applications to federated learning and analytics. We provide a tight analysis of its privacy guarantees, showing that it achieves the same privacy-accuracy trade-offs as the continuous Gaussian mechanism. Our analysis is based on a novel bound on the R\'enyi divergence of two Poisson binomial distributions that may be of independent interest. Unlike previous discrete DP schemes based on additive noise, our mechanism encodes local information into a parameter of the binomial distribution, and hence the output distribution is discrete with bounded support. Moreover, the support does not increase as the privacy budget $\varepsilon \rightarrow 0$ as in the case of additive schemes which require the addition of more noise to achieve higher privacy; on the contrary, the support becomes smaller as $\varepsilon \rightarrow 0$. The bounded support enables us to combine our mechanism with secure aggregation (SecAgg), a multi-party cryptographic protocol, without the need of performing modular clipping which results in an unbiased estimator of the sum of the local vectors. This in turn allows us to apply it in the private FL setting and provide an upper bound on the convergence rate of the SGD algorithm. Moreover, since the support of the output distribution becomes smaller as $\varepsilon \rightarrow 0$, the communication cost of our scheme decreases with the privacy constraint $\varepsilon$, outperforming all previous distributed DP schemes based on additive noise in the high privacy or low communication regimes.