A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

TL;DR

This paper introduces a majorized-generalized ADMM for convex composite programming that handles nonsmooth+smooth objectives without quadratic assumptions, ensuring convergence and demonstrating effectiveness in applications.

Contribution

It develops a new ADMM variant using majorization to efficiently solve non-quadratic convex composite problems with proven convergence.

Findings

Algorithm converges globally under certain conditions.

Effective in simulated convex composite optimization.

Shows promising results in sparse inverse covariance estimation.

Abstract

The linearly constrained convex composite programming problems whose objective function contains two blocks with each block being the form of nonsmooth+smooth arises frequently in multiple fields of applications. If both of the smooth terms are quadratic, this problem can be solved efficiently by using the symmetric Gaussian-Seidel (sGS) technique based proximal alternating direction method of multipliers (ADMM). However, in the non-quadratic case, the sGS technique can not be used any more, which leads to the separable structure of nonsmooth+smooth had to be ignored. In this paper, we present a generalized ADMM and particularly use a majorization technique to make the corresponding subproblems more amenable to efficient computations. Under some appropriate conditions, we prove its global convergence for the relaxation factor in $(0,2)$. We apply the algorithm to solve a kind of simulated convex composite optimization problems and a type of sparse inverse covariance matrix estimation problems which illustrates that the effectiveness of the algorithm are obvious.