Golden ratio primal-dual algorithm with linesearch
Xiaokai Chang, Junfeng Yang, Hongchao Zhang

TL;DR
This paper introduces a linesearch-enhanced Golden Ratio Primal-Dual Algorithm (GRPDA) that adaptively adjusts stepsizes without spectral norm knowledge, achieving improved convergence rates for convex optimization problems.
Contribution
The paper proposes a novel linesearch strategy for GRPDA, enabling larger stepsizes, adaptive parameter selection, and faster convergence rates, with theoretical guarantees and practical efficiency.
Findings
Global convergence with ${ m O}(1/N)$ rate
Faster ${ m O}(1/N^2)$ rate when one function is strongly convex
Linear convergence when both functions are strongly convex
Abstract
Golden ratio primal-dual algorithm (GRPDA) is a new variant of the classical Arrow-Hurwicz method for solving structured convex optimization problem, in which the objective function consists of the sum of two closed proper convex functions, one of which involves a composition with a linear transform. In this paper, we propose a linesearch strategy for GRPDA, which not only does not require the spectral norm of the linear transform but also allows adaptive and potentially much larger stepsizes. Within each linesearch step, only the dual variable needs to be updated, and it is thus quite cheap and does not require any extra matrix-vector multiplications for many special yet important applications, e.g., regularized least squares problem. Global convergence and ergodic convergence rate results measured by the primal-dual gap function are established, where denotes the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Optimization Algorithms Research · Optimization and Variational Analysis · Sparse and Compressive Sensing Techniques
