Linear Convergence Rate Analysis of Proximal Generalized ADMM for Convex Composite Programming

TL;DR

This paper proves that the proximal generalized ADMM algorithm converges linearly for convex composite problems under mild conditions, providing theoretical guarantees for its efficiency.

Contribution

It establishes the first linear convergence rate analysis for p-GADMM, addressing a gap in the theoretical understanding of this method.

Findings

p-GADMM exhibits Q-linear convergence under mild conditions.

Proximal terms need to be positive definite, which is common in practice.

The convergence rate analysis accounts for relaxed points in the algorithm.

Abstract

The proximal generalized alternating direction method of multipliers (p-GADMM) is substantially efficient for solving convex composite programming problems of high-dimensional to moderate accuracy. The global convergence of this method was established by Xiao, Chen & Li [Math. Program. Comput., 2018], but its convergence rate was not given. One may take it for granted that the convergence rate could be proved easily by mimicking the proximal ADMM, but we find the relaxed points will certainly cause many difficulties for theoretical analysis. In this paper, we devote to exploring its convergence behavior and show that the sequence generated by p-GADMM possesses Q-linear convergence rate under some mild conditions. We would like to note that the proximal terms at the subproblems are required to be positive definite, which is very common in most practical implementations although it seems to be a bit strong.