A Note on the Convergence of ADMM for Linearly Constrained Convex Optimization Problems

TL;DR

This paper challenges previous assumptions about ADMM convergence by providing a counterexample and establishing new mild conditions that ensure convergence in a more general semi-proximal setting with larger step-lengths.

Contribution

It constructs a counterexample to prior convergence claims and introduces mild conditions for convergence of semi-proximal ADMM with larger step-lengths.

Findings

Counterexample shows previous convergence claims can be false without solution existence assumptions

Mild conditions are identified that guarantee convergence of semi-proximal ADMM

Larger step-lengths, even exceeding the golden ratio, can be used effectively

Abstract

This note serves two purposes. Firstly, we construct a counterexample to show that the statement on the convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex optimization problems in a highly influential paper by Boyd et al. [Found. Trends Mach. Learn. 3(1) 1-122 (2011)] can be false if no prior condition on the existence of solutions to all the subproblems involved is assumed to hold. Secondly, we present fairly mild conditions to guarantee the existence of solutions to all the subproblems and provide a rigorous convergence analysis on the ADMM, under a more general and useful semi-proximal ADMM (sPADMM) setting considered by Fazel et al. [SIAM J. Matrix Anal. Appl. 34(3) 946-977 (2013)], with a computationally more attractive large step-length that can even exceed the practically much preferred golden ratio of $(1+\sqrt{5})/2$.