Analysis of Fully Preconditioned ADMM with Relaxation in Hilbert Spaces

TL;DR

This paper proves the weak convergence of a generalized preconditioned and relaxed ADMM in infinite-dimensional Hilbert spaces, establishing ergodic convergence rates and exploring connections to Douglas-Rachford splitting methods.

Contribution

It provides the first weak convergence proof for preconditioned and relaxed ADMM in infinite-dimensional Hilbert spaces with mild conditions.

Findings

Weak convergence of preconditioned and relaxed ADMM established

Ergodic convergence rates derived for the method

Numerical tests demonstrate efficiency of the proposed variants

Abstract

Alternating direction method of multipliers (ADMM) is a powerful first order methods for various applications in signal processing and imaging. However, there is no clear result on the weak convergence of ADMM with relaxation studied by Eckstein and Bertsakas \cite{EP} in infinite dimensional Hilbert spaces. In this paper, by employing a kind of "partial" gap analysis, we prove the weak convergence of general preconditioned and relaxed ADMM in infinite dimensional Hilbert spaces, with preconditioning for solving all the involved implicit equations under mild conditions. We also give the corresponding ergodic convergence rates respecting to the "partial" gap function. Furthermore, the connections between certain preconditioned and relaxed ADMM and the corresponding Douglas-Rachford splitting methods are also discussed, following the idea of Gabay in \cite{DGBA}. Numerical tests also show the efficiency of the proposed overrelaxation variants of preconditioned ADMM.