On the Convergence of Primal-Dual Proximal Incremental Aggregated Gradient Algorithms
Zhou Xianchen, Peng Wei, Wang Hongxia

TL;DR
This paper introduces primal-dual proximal incremental aggregated gradient methods for saddle point problems, proving sublinear and linear convergence under various conditions, and extends these methods with and without extrapolation.
Contribution
It adapts proximal incremental aggregated gradient methods to saddle point problems and establishes convergence properties, including linear convergence under strong convexity.
Findings
Primal-dual gap converges sublinearly to 0.
Iteration sequence converges to a saddle point.
Linear convergence under strong convexity conditions.
Abstract
In this paper, we adapt proximal incremental aggregated gradient methods to saddle point problems, which is motivated by decoupling linear transformations in regularized empirical risk minimization models. First, the Primal-Dual Proximal Incremental Aggregated (PD-PIAG) methods with extrapolations were proposed. We proved that the primal-dual gap of the averaged iteration sequence sublinearly converges to 0, and the iteration sequence converges to some saddle point. Under the strong convexity of and , we proved that the iteration sequence linearly converges to the saddle point. Then, we propose a PD-PIAG method without extrapolations. The primal-dual gap of the iteration sequence is proved to be sublinearly convergent under strong convexity of .
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Stochastic Gradient Optimization Techniques · Numerical methods in inverse problems
