AdaCliP: Adaptive Clipping for Private SGD
Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu, and Sashank J. Reddi, Sanjiv Kumar

TL;DR
AdaCliP introduces an adaptive clipping method for differentially private SGD that reduces noise addition and improves model accuracy, advancing privacy-preserving machine learning techniques.
Contribution
It proposes a novel coordinate-wise adaptive clipping approach for DP-SGD, providing theoretical guarantees and empirical improvements over existing methods.
Findings
Reduces noise in DP-SGD compared to previous methods.
Produces models with higher accuracy under privacy constraints.
Theoretically guarantees adaptive clipping benefits.
Abstract
Privacy preserving machine learning algorithms are crucial for learning models over user data to protect sensitive information. Motivated by this, differentially private stochastic gradient descent (SGD) algorithms for training machine learning models have been proposed. At each step, these algorithms modify the gradients and add noise proportional to the sensitivity of the modified gradients. Under this framework, we propose AdaCliP, a theoretically motivated differentially private SGD algorithm that provably adds less noise compared to the previous methods, by using coordinate-wise adaptive clipping of the gradient. We empirically demonstrate that AdaCliP reduces the amount of added noise and produces models with better accuracy.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Stochastic Gradient Optimization Techniques · Cryptography and Data Security
MethodsStochastic Gradient Descent
