The global linear convergence rate of the proximal version of the generalized alternating direction method of multipliers for separable convex programming

TL;DR

This paper proves the global linear convergence rate of the linearized and doubly linearized generalized alternating direction method of multipliers (GADMM) for separable convex programming, extending previous convergence results.

Contribution

It introduces the doubly linearized GADMM and establishes its linear convergence rate, generalizing and improving existing convergence results for these methods.

Findings

L-GADMM has a linear convergence rate under certain conditions.

DL-GADMM also exhibits linear convergence for functions with piecewise linear subdifferentials.

The results extend known convergence properties of GADMM variants.

Abstract

To solve the separable convex optimization problem with linear constraints, Eckstein and Bertsekas introduced the generalized alternating direction method of multipliers (in short, GADMM), which is an efficient and simple acceleration scheme of the aternating direction method of multipliers. Recently, \textbf{Fang et. al} proposed the linearized version of generalized alternating direction method of multipliers (in short, L-GADMM), where one of its subproblems is approximated by a linearization strategy, and proved its worst-case $\mathcal{O}(1/t)$ convergence rate measured by the iteration complexity in both ergodic and nonergodic senses. In this paper, we introduce the doubly linearized version of generalized alternating direction method of multipliers (in short, DL-GADMM), where both the $x$-subproblem and $y$-subproblem are approximated by linearization strategies. Based on the error bound approach, we establish the linear convergence rate of both L-GADMM and DL-GADMM for separable convex optimization problem that the subdifferentials of the underlying functions are piecewise linear multifunctions. The results in this paper extend, generalize and improve some known results in the literature.