New convergence analysis of a primal-dual algorithm with large stepsizes

TL;DR

This paper provides a new, weaker convergence condition for a primal-dual algorithm used in convex optimization, demonstrating linear convergence under certain assumptions and applying the results to decentralized consensus algorithms.

Contribution

It establishes a weaker convergence condition for the primal-dual algorithm and proves its linear convergence, also applying the findings to decentralized consensus methods.

Findings

Convergence under weaker stepsize conditions.

Linear convergence with additional assumptions.

Application to decentralized consensus algorithms.

Abstract

We consider a primal-dual algorithm for minimizing $f(x)+h\square l(Ax)$ with Fr\'echet differentiable $f$ and $l^*$. This primal-dual algorithm has two names in literature: Primal-Dual Fixed-Point algorithm based on the Proximity Operator (PDFP$^2$O) and Proximal Alternating Predictor-Corrector (PAPC). In this paper, we prove its convergence under a weaker condition on the stepsizes than existing ones. With additional assumptions, we show its linear convergence. In addition, we show that this condition (the upper bound of the stepsize) is tight and can not be weakened. This result also recovers a recently proposed positive-indefinite linearized augmented Lagrangian method. In addition, we apply this result to a decentralized consensus algorithm PG-EXTRA and derive the weakest convergence condition.