cpSGD: Communication-efficient and differentially-private distributed SGD
Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, H., Brendan Mcmahan

TL;DR
This paper introduces cpSGD, a distributed stochastic gradient descent algorithm that simultaneously achieves communication efficiency and differential privacy, suitable for mobile device clients in distributed learning.
Contribution
The paper proposes a novel algorithm, cpSGD, that combines communication efficiency with differential privacy guarantees in distributed SGD settings.
Findings
Uses O(log log(nd)) bits per client per coordinate
Ensures constant privacy across clients
Improves analysis of the Binomial mechanism for privacy and utility
Abstract
Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication cost or introducing privacy guarantees, but none of the proposed communication efficient methods are known to be privacy preserving and none of the known privacy mechanisms are known to be communication efficient. To this end, we study algorithms that achieve both communication efficiency and differential privacy. For variables and clients, the proposed method uses bits of communication per client per coordinate and ensures constant privacy. We also extend and improve previous analysis of the \emph{Binomial mechanism} showing that it…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Cryptography and Data Security · Cooperative Communication and Network Coding
