Adaptive primal dual hybrid gradient algorithms based on average spectrum for saddle point problems

TL;DR

This paper introduces adaptive primal dual hybrid gradient algorithms that leverage average spectrum analysis to improve convergence and performance in saddle point problems, overcoming limitations of existing methods.

Contribution

The paper proposes a new class of adaptive PDHG algorithms based on average spectrum, with proven convergence and potential acceleration for convex saddle point problems.

Findings

Demonstrates superior numerical performance on assignment problems.

Provides convergence theory based on average spectrum rather than restrictive conditions.

Shows potential for acceleration over traditional PDHG methods.

Abstract

The primal dual hybrid gradient algorithm (PDHG), which is also known as the Arrow-Hurwicz method, is a fundamental algorithm for saddle point problems especially in imaging. It also inspires a great number of influential algorithms such as the stochastic PDHG and the Chambolle-Pock's primal dual algorithm. In the literature, convergence theory of the PDHG is established only when some more restrictive conditions are additionally assumed, and it is proved that the PDHG with any constant step sizes could diverge for generic setting of convex saddle point problems. The Chambolle-Pock's primal dual algorithm, as an influential variant of the PDHG, is thus widely used due to its provable convergence theory and competitive numerical performance. However, step sizes of the Chambolle-Pock's primal dual algorithm are inherently bounded by its associated matrix spectrum, and this restriction could limit its computational capacity structurally. To address these limitations both in theory and practice, we propose a class of adaptive primal dual hybrid gradient algorithms for generic convex saddle point problems in this paper. By exploiting the prediction-correction algorithmic framework, the global convergence theory of the proposed schemes can be determined only by the average spectrum of the underlying matrix, and it thus leads to a potential acceleration. The numerical experiment on the assignment problem illustrates the superior numerical performance of the proposed method.