GeoClip: Geometry-Aware Clipping for Differentially Private SGD
Atefeh Gilani, Naima Tasnim, Lalitha Sankar, Oliver Kosut

TL;DR
GeoClip introduces a geometry-aware gradient clipping method for DP-SGD that adaptively aligns with gradient distribution geometry, improving privacy-utility trade-offs without extra privacy cost.
Contribution
It proposes GeoClip, a novel framework that clips and perturbs gradients in a transformed basis aligned with their geometry, with convergence guarantees and optimal transformation derivation.
Findings
GeoClip outperforms existing adaptive clipping methods under the same privacy budget.
GeoClip effectively estimates the transformation using only noisy gradients.
Experiments on tabular and image datasets validate GeoClip's superior performance.
Abstract
Differentially private stochastic gradient descent (DP-SGD) is the most widely used method for training machine learning models with provable privacy guarantees. A key challenge in DP-SGD is setting the per-sample gradient clipping threshold, which significantly affects the trade-off between privacy and utility. While recent adaptive methods improve performance by adjusting this threshold during training, they operate in the standard coordinate system and fail to account for correlations across the coordinates of the gradient. We propose GeoClip, a geometry-aware framework that clips and perturbs gradients in a transformed basis aligned with the geometry of the gradient distribution. GeoClip adaptively estimates this transformation using only previously released noisy gradients, incurring no additional privacy cost. We provide convergence guarantees for GeoClip and derive a closed-form…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Stochastic Gradient Optimization Techniques · Adversarial Robustness in Machine Learning
MethodsGradient Clipping
