Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

TL;DR

This paper presents a convergence analysis of a generalized ADMM method enhanced with majorization for linearly constrained composite convex problems, addressing limitations with non-quadratic smooth functions and demonstrating improved efficiency through numerical experiments.

Contribution

It introduces a majorized technique to extend ADMM's applicability to non-quadratic smooth functions and proves its global convergence.

Findings

The proposed method converges globally to a KKT point.

Numerical experiments show improved efficiency over existing methods.

The approach effectively decomposes complex subproblems into smaller ones.

Abstract

The generalized alternating direction method of multipliers (ADMM) of Xiao et al. [{\tt Math. Prog. Comput., 2018}] aims at the two-block linearly constrained composite convex programming problem, in which each block is in the form of "nonsmooth + quadratic". However, in the case of non-quadratic (but smooth), this method may fail unless the favorable structure of 'nonsmooth + smooth' is no longer used. This paper aims to remedy this defect by using a majorized technique to approximate the augmented Lagrangian function, so that the corresponding subprobllem can be decomposed into some smaller problems and then solved separately. Furthermore, the recent symmetric Gauss-Seidel (sGS) decomposition theorem guarantees the equivalence between the bigger subproblem and these smaller ones. This paper focuses on convergence analysis, that is, we prove that the sequence generated by the proposed method converges globally to a Karush-Kuhn-Tucker point of the considered problem. Finally, we do some numerical experiments on a kind of simulated convex composite optimization problems which illustrate that the proposed method is more efficient than its compared ones.