A General Analysis of the Convergence of ADMM

TL;DR

This paper presents a new, general proof of ADMM's linear convergence under strong convexity, using a dynamical systems framework, and offers practical parameter selection guidance.

Contribution

It introduces a unified proof technique for ADMM convergence that removes parameter restrictions and provides bounds for practical parameter tuning.

Findings

New proof of linear convergence for ADMM with strong convexity

A framework that generalizes existing convergence results

Practical bounds for selecting algorithm parameters

Abstract

We provide a new proof of the linear convergence of the alternating direction method of multipliers (ADMM) when one of the objective terms is strongly convex. Our proof is based on a framework for analyzing optimization algorithms introduced in Lessard et al. (2014), reducing algorithm convergence to verifying the stability of a dynamical system. This approach generalizes a number of existing results and obviates any assumptions about specific choices of algorithm parameters. On a numerical example, we demonstrate that minimizing the derived bound on the convergence rate provides a practical approach to selecting algorithm parameters for particular ADMM instances. We complement our upper bound by constructing a nearly-matching lower bound on the worst-case rate of convergence.