Accelerated ADMM: Automated Parameter Tuning and Improved Linear Convergence

TL;DR

This paper analyzes an accelerated ADMM scheme for strongly convex problems, providing new convergence bounds, parameter tuning heuristics, and empirical validation demonstrating improved linear convergence over existing methods.

Contribution

It introduces a novel Lur'e system framework for accelerated ADMM, deriving improved convergence bounds and practical parameter tuning heuristics.

Findings

Accelerated ADMM achieves faster linear convergence rates.

Parameter tuning heuristics significantly impact convergence speed.

Empirical results validate theoretical improvements on LASSO regression.

Abstract

This work studies the linear convergence of an accelerated scheme of the Alternating Direction Method of Multipliers (ADMM) for strongly convex and Lipschitz-smooth problems. We use the methodology of expressing the accelerated ADMM as a Lur'e system, i.e., an interconnection of a linear dynamical system in feedback with a slope-restricted operator, and we use Integral Quadratic Constraints to establish linear convergence. In addition, we propose several parameter tuning heuristics and their impact on the convergence rate through numerical analyses. Our new bounds show improved linear convergence rates compared to the vanilla algorithm and previous proposed accelerated variants, which is also empirically validated on a LASSO regression benchmark.