Activity Identification and Local Linear Convergence of Douglas--Rachford/ADMM under Partial Smoothness

TL;DR

This paper analyzes the local convergence of Douglas--Rachford and ADMM algorithms in convex optimization, showing finite manifold identification and linear convergence under partial smoothness assumptions, with practical illustrations.

Contribution

It provides new theoretical insights into the manifold identification and local linear convergence of DR and ADMM under partial smoothness conditions.

Findings

Finite-time manifold identification by DR and ADMM.

Local linear convergence when manifolds are affine or linear.

Convergence radius characterized by Friedrichs angle for polyhedral functions.

Abstract

Convex optimization has become ubiquitous in most quantitative disciplines of science, including variational image processing. Proximal splitting algorithms are becoming popular to solve such structured convex optimization problems. Within this class of algorithms, Douglas--Rachford (DR) and alternating direction method of multipliers (ADMM) are designed to minimize the sum of two proper lower semi-continuous convex functions whose proximity operators are easy to compute. The goal of this work is to understand the local convergence behaviour of DR (resp. ADMM) when the involved functions (resp. their Legendre-Fenchel conjugates) are moreover partly smooth. More precisely, when both of the two functions (resp. their conjugates) are partly smooth relative to their respective manifolds, we show that DR (resp. ADMM) identifies these manifolds in finite time. Moreover, when these manifolds are affine or linear, we prove that DR/ADMM is locally linearly convergent. When $J$ and $G$ are locally polyhedral, we show that the optimal convergence radius is given in terms of the cosine of the Friedrichs angle between the tangent spaces of the identified manifolds. This is illustrated by several concrete examples and supported by numerical experiments.