Iterative Complexity and Its Applications of Linearized Generalized Alternating Direction Method of Multipliers with Multi-block Case

TL;DR

This paper extends the linearized generalized alternating direction method of multipliers (L-GADMM) to multi-block convex optimization problems, proving its convergence and demonstrating its efficiency through numerical experiments.

Contribution

It introduces a multi-block extension of L-GADMM, proves its global convergence, and establishes its convergence rate, filling a gap in multi-block convex optimization methods.

Findings

Proved global convergence of the multi-block L-GADMM.

Established worst-case convergence rates in ergodic and nonergodic senses.

Demonstrated efficiency through numerical experiments on correlation matrix calibration.

Abstract

We consider a multi-block separable convex optimization problem with the linear constraints, where the objective function is the sum of m individual convex functions without overlapping variables. The linearized version of the generalized alternating direction method of multipliers (L-GADMM) is particularly efficient for the two-block separable convex programming problem and its convergence was proved when two blocks of variables are alternatively updated. However,the convergence and some practical applications of the extension (m>=3) of the L-GADMM is still in its infancy. In this paper, we extend this algorithm to the general case where the objective function consists of the sum of m-block convex functions. Theoretically, we prove global convergence of the new method and establish the worst-case convergence rate in the ergodic and nonergodic senses for the proposed algorithm. The efficiency of the new method is further demonstrated through numerical results on the calibration of the correlation matrices.