Eventual linear convergence of the Douglas Rachford iteration for basis pursuit

TL;DR

This paper analyzes the linear convergence rate of the Douglas-Rachford algorithm for basis pursuit, relating it to principal angles and restricted isometry constants, and explores parameter choices for acceleration.

Contribution

It provides a simple analysis of the Douglas-Rachford splitting algorithm's convergence in basis pursuit and extends the analysis to regularized and over-relaxed schemes.

Findings

Convergence rate is quantified in terms of principal angles.

Bounded the rate using restricted isometry constants in compressed sensing.

Guidelines for parameter selection to accelerate convergence.

Abstract

We provide a simple analysis of the Douglas-Rachford splitting algorithm in the context of $\ell^1$ minimization with linear constraints, and quantify the asymptotic linear convergence rate in terms of principal angles between relevant vector spaces. In the compressed sensing setting, we show how to bound this rate in terms of the restricted isometry constant. More general iterative schemes obtained by $\ell^2$-regularization and over-relaxation including the dual split Bregman method are also treated, which answers the question how to choose the relaxation and soft-thresholding parameters to accelerate the asymptotic convergence rate. We make no attempt at characterizing the transient regime preceding the onset of linear convergence.