Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence results

TL;DR

This paper establishes tight convergence guarantees for Douglas-Rachford splitting, ADMM, and Peaceman-Rachford splitting in nonconvex optimization by leveraging the Douglas-Rachford envelope, broadening their applicability.

Contribution

It introduces a unified framework using the Douglas-Rachford envelope to derive less restrictive, tight global convergence results for these algorithms in nonconvex settings.

Findings

Convergence bounds are tight for over-relaxation parameters in (0,2].

The analysis applies under less restrictive conditions than previous results.

A primal equivalence between ADMM and DRS is established, generalizing known dualities.

Abstract

Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising global convergence guarantees for ADMM, DRS and PRS applied to nonconvex problems under less restrictive conditions, larger prox-stepsizes and over-relaxation parameters than previously known. In fact, our bounds are tight whenever the over-relaxation parameter ranges in $(0,2]$. The analysis of ADMM uses a universal primal equivalence with DRS that generalizes the known duality of the algorithms.