Adaptive Primal-Dual Hybrid Gradient Methods for Saddle-Point Problems

TL;DR

This paper introduces adaptive PDHG algorithms that automatically tune stepsize parameters, ensuring fast convergence and ease of use, supported by theoretical convergence proofs and numerical experiments.

Contribution

The paper presents novel adaptive PDHG schemes with proven convergence, eliminating the need for manual stepsize tuning in saddle-point optimization problems.

Findings

Adaptive schemes outperform non-adaptive methods in efficiency.

The methods are simple to implement and require no user parameter tuning.

Numerical experiments confirm faster convergence rates.

Abstract

The Primal-Dual hybrid gradient (PDHG) method is a powerful optimization scheme that breaks complex problems into simple sub-steps. Unfortunately, PDHG methods require the user to choose stepsize parameters, and the speed of convergence is highly sensitive to this choice. We introduce new adaptive PDHG schemes that automatically tune the stepsize parameters for fast convergence without user inputs. We prove rigorous convergence results for our methods, and identify the conditions required for convergence. We also develop practical implementations of adaptive schemes that formally satisfy the convergence requirements. Numerical experiments show that adaptive PDHG methods have advantages over non-adaptive implementations in terms of both efficiency and simplicity for the user.