The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle

TL;DR

This paper analyzes the Douglas-Rachford algorithm for two subspaces, proving strong convergence to the projection point and establishing a linear convergence rate equal to the cosine of the Friedrichs angle, applicable in infinite-dimensional spaces.

Contribution

It precisely characterizes the convergence behavior of the Douglas-Rachford algorithm for subspaces, including the convergence rate and the limit location, extending results to infinite-dimensional Hilbert spaces.

Findings

Converges strongly to the projection of the initial point onto the intersection.

Achieves linear convergence rate equal to the cosine of the Friedrichs angle.

Applicable to possibly infinite-dimensional Hilbert spaces.

Abstract

The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings. In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.