Solving Multiple-Block Separable Convex Minimization Problems Using Two-Block Alternating Direction Method of Multipliers

TL;DR

This paper proposes a strategy to solve multi-block separable convex minimization problems by transforming them into two-block problems and applying ADMM, with proven convergence and improved efficiency over standard multi-block ADMM.

Contribution

It introduces a new approach transforming multi-block problems into two-block problems for ADMM, providing convergence analysis and demonstrating numerical advantages.

Findings

Convergence results with improved O(1/ε) iteration complexity.

Numerical experiments show the approach outperforms standard multi-block ADMM.

The method effectively solves problems like basis pursuit and PCA.

Abstract

In this paper, we consider solving multiple-block separable convex minimization problems using alternating direction method of multipliers (ADMM). Motivated by the fact that the existing convergence theory for ADMM is mostly limited to the two-block case, we analyze in this paper, both theoretically and numerically, a new strategy that first transforms a multi-block problem into an equivalent two-block problem (either in the primal domain or in the dual domain) and then solves it using the standard two-block ADMM. In particular, we derive convergence results for this two-block ADMM approach to solve multi-block separable convex minimization problems, including an improved O(1/\epsilon) iteration complexity result. Moreover, we compare the numerical efficiency of this approach with the standard multi-block ADMM on several separable convex minimization problems which include basis pursuit, robust principal component analysis and latent variable Gaussian graphical model selection. The numerical results show that the multiple-block ADMM, although lacks theoretical convergence guarantees, typically outperforms two-block ADMMs.