Faster convergence rates of relaxed Peaceman-Rachford and ADMM under regularity assumptions

TL;DR

This paper analyzes the convergence rates of splitting algorithms like PRS and ADMM, showing they adapt to problem regularity and improve upon worst-case rates using simple techniques.

Contribution

It provides a comprehensive convergence rate analysis of PRS, DRS, and ADMM under various regularity assumptions, highlighting their adaptive behavior.

Findings

Relaxed PRS and ADMM adapt to problem regularity.

Improved convergence rates under strong convexity and Lipschitz differentiability.

Results obtained with simple analytical techniques.

Abstract

Splitting schemes are a class of powerful algorithms that solve complicated monotone inclusion and convex optimization problems that are built from many simpler pieces. They give rise to algorithms in which the simple pieces of the decomposition are processed individually. This leads to easily implementable and highly parallelizable algorithms, which often obtain nearly state-of-the-art performance. In this paper, we provide a comprehensive convergence rate analysis of the Douglas-Rachford splitting (DRS), Peaceman-Rachford splitting (PRS), and alternating direction method of multipliers (ADMM) algorithms under various regularity assumptions including strong convexity, Lipschitz differentiability, and bounded linear regularity. The main consequence of this work is that relaxed PRS and ADMM automatically adapt to the regularity of the problem and achieve convergence rates that improve upon the (tight) worst-case rates that hold in the absence of such regularity. All of the results are obtained using simple techniques.