Differentially Private Optimization for Smooth Nonconvex ERM
Changyu Gao, Stephen J. Wright
TL;DR
This paper introduces differentially private optimization algorithms tailored for smooth nonconvex empirical risk minimization, emphasizing efficiency and practical applicability through innovative strategies.
Contribution
It presents simple, effective differentially private algorithms that incorporate line search, mini-batching, and a two-phase approach to improve convergence and practicality.
Findings
Algorithms successfully find approximate second-order solutions.
Numerical experiments confirm improved speed and effectiveness.
Approach enhances privacy-preserving nonconvex optimization.
Abstract
We develop simple differentially private optimization algorithms that move along directions of (expected) descent to find an approximate second-order solution for nonconvex ERM. We use line search, mini-batching, and a two-phase strategy to improve the speed and practicality of the algorithm. Numerical experiments demonstrate the effectiveness of these approaches.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCryptography and Data Security · Stochastic Gradient Optimization Techniques · Polynomial and algebraic computation
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings