Tuning Over-Relaxed ADMM

TL;DR

This paper summarizes the use of IQC framework to analyze over-relaxed ADMM convergence and derives simple formulas for optimal parameter selection applicable to strongly convex functions.

Contribution

It provides a summary of the IQC-based convergence bounds for over-relaxed ADMM and introduces explicit formulas for optimal parameters.

Findings

Explicit formulas for optimal over-relaxation parameters

Convergence bounds valid for strongly convex functions

Simplified analysis of over-relaxed ADMM performance

Abstract

The framework of Integral Quadratic Constraints (IQC) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to a semi-definite program (SDP). In the case of over-relaxed Alternating Direction Method of Multipliers (ADMM), an explicit and closed form solution to this SDP was derived in our recent work [1]. The purpose of this paper is twofold. First, we summarize these results. Second, we explore one of its consequences which allows us to obtain general and simple formulas for optimal parameter selection. These results are valid for arbitrary strongly convex objective functions.