# Generalized Symmetric ADMM for Separable Convex Optimization

## TL;DR

This paper introduces a Generalized Symmetric ADMM that updates dual variables twice with larger stepsizes, improving convergence and flexibility for multi-block convex optimization, especially suitable for big data applications.

## Contribution

The paper proposes a novel GS-ADMM algorithm with larger stepsize domain, combining Gauss-Seidel and Jacobi updates, and provides convergence guarantees and practical effectiveness.

## Key findings

- Global convergence with $O(1/t)$ rate established.
- Larger stepsize domain compared to existing methods.
- Effective in sparse matrix minimization for statistical learning.

## Abstract

The Alternating Direction Method of Multipliers (ADMM) has been proved to be effective for solving separable convex optimization subject to linear constraints. In this paper, we propose a Generalized Symmetric ADMM (GS-ADMM), which updates the Lagrange multiplier twice with suitable stepsizes, to solve the multi-block separable convex programming. This GS-ADMM partitions the data into two group variables so that one group consists of $p$ block variables while the other has $q$ block variables, where $p \ge 1$ and $q \ge 1$ are two integers. The two grouped variables are updated in a {\it Gauss-Seidel} scheme, while the variables within each group are updated in a {\it Jacobi} scheme, which would make it very attractive for a big data setting. By adding proper proximal terms to the subproblems, we specify the domain of the stepsizes to guarantee that GS-ADMM is globally convergent with a worst-case $O(1/t)$ ergodic convergence rate. It turns out that our convergence domain of the stepsizes is significantly larger than other convergence domains in the literature. Hence, the GS-ADMM is more flexible and attractive on choosing and using larger stepsizes of the dual variable. Besides, two special cases of GS-ADMM, which allows using zero penalty terms, are also discussed and analyzed. Compared with several state-of-the-art methods, preliminary numerical experiments on solving a sparse matrix minimization problem in the statistical learning show that our proposed method is effective and promising.