ADMM and Accelerated ADMM as Continuous Dynamical Systems

TL;DR

This paper models ADMM and its accelerated variant as continuous dynamical systems, analyzing their stability and convergence using Lyapunov methods to deepen understanding of their behavior.

Contribution

It derives the differential equations representing the continuous limits of ADMM and accelerated ADMM, providing new insights into their stability and convergence properties.

Findings

Differential equations for ADMM and accelerated ADMM derived

Lyapunov stability analysis applied to these dynamical systems

Convergence rates established for the continuous models

Abstract

Recently, there has been an increasing interest in using tools from dynamical systems to analyze the behavior of simple optimization algorithms such as gradient descent and accelerated variants. This paper strengthens such connections by deriving the differential equations that model the continuous limit of the sequence of iterates generated by the alternating direction method of multipliers, as well as an accelerated variant. We employ the direct method of Lyapunov to analyze the stability of critical points of the dynamical systems and to obtain associated convergence rates.