Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence

TL;DR

This paper develops an adaptive stepsize rule for the Douglas-Rachford method, improving its practical performance and eliminating the need for manual tuning, with proven convergence and applications to ADMM.

Contribution

It introduces a novel adaptive stepsize strategy for non-stationary Douglas-Rachford and ADMM methods, with theoretical convergence guarantees.

Findings

The adaptive stepsize rule eliminates the need for manual tuning.

Convergence is proven for non-stationary schemes with summable stepsize increments.

Numerical examples demonstrate the efficiency of the proposed methods.

Abstract

We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at developing an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for stepsize tuning. We analyze a general non-stationary DR scheme and prove its convergence for a convergent sequence of stepsizes with summable increments. This, in turn, proves the convergence of the method with the new adaptive stepsize rule. We also derive the related non-stationary alternating direction method of multipliers (ADMM) from such a non-stationary DR method. We illustrate the efficiency of the proposed methods on several numerical examples.