A note on alternating projections in Hilbert space

TL;DR

This paper offers a direct proof of the asymptotic behavior of alternating projections onto convex sets in Hilbert space, utilizing nonexpansive mapping theory to clarify convergence properties.

Contribution

It presents a new, direct proof of the asymptotic behavior of alternating projections in Hilbert spaces, enhancing understanding with a nonexpansive mapping approach.

Findings

Proof of asymptotic convergence of alternating projections

Application of nonexpansive mapping theory to projection analysis

Clarification of convergence behavior in Hilbert spaces

Abstract

We provide a direct proof of a result regarding the asymptotic behavior of alternating nearest point projections onto two closed and convex sets in a Hilbert space. Our arguments are based on nonexpansive mapping theory.

Equations

$$x_{2n+1}=P_1{x}_{2n}\text{and }{x}_{2n+2}=P_2{x}_{2n+1},$$

$$\mathrm{d}(C_1,C_2):=\inf \{|x-y|:x\in C_1,y\in C_2\}.$$

$$\underset{n\to \infty }{\lim }|x_{2n+2}-x_{2n+1}|=\underset{n\to \infty }{\lim }|x_{2n+1}-x_{2n}|=\mathrm{d}(C_1,C_2).$$

$$|x_{2n+2}-x_{2n+1}|\le |x_{2n+1}-x_{2n}|\le |x_{2n}-x_{2n-1}|\le |x_{2n-1}-x_{2n-2}|$$

$$\underset{n\to \infty }{\lim }(x_{2n+2}-x_{2n+1})=\underset{n\to \infty }{\lim }(x_{2n}-x_{2n+1})=v,$$

$$|y_n+v{|}^2+|y_n-v{|}^2=2(|y_n{|}^2+|v{|}^2)$$

$$|(y_n-v)/2{|}^2=(|y_n{|}^2+|v{|}^2)/2-|(y_n+v)/2{|}^2.$$

$$|y_n-v{|}^2\le 2(|y_n{|}^2-|v{|}^2)$$

$$|y-P_{C}x{|}^2+|x-P_{C}x{|}^2\le |x-y{|}^2$$

$$\begin{array}{rl}|x-y{|}^2& =|x-P_{C}x+P_{C}x-y{|}^2\\ & =|x-P_{C}x{|}^2+2⟨x-P_{C}x,P_{C}x-y⟩+|y-P_{C}x{|}^2\\ & \ge |x-P_{C}x{|}^2+|y-P_{C}x{|}^2,\end{array}$$

$$\mathrm{F}(P_2{P}_1)=\{b\in C_2:|b-P_{1}b|=\mathrm{d}(C_1,C_2)\}.$$

$$|x_{2n+2}-z|=|P_2{x}_{2n+1}-P_2{P}_{1}z|\le |x_{2n+1}-P_{1}z|=|P_1{x}_{2n}-P_{1}z|\le |x_{2n}-z|$$

$$\underset{n\to \infty }{\lim }(|x_{2n+1}-P_{1}z|-|P_2{x}_{2n+1}-P_2{P}_{1}z|)=0.$$

$$\underset{n\to \infty }{\lim }(x_{2n+1}-x_{2n+2})=P_{1}z-z.$$

$$\underset{n\to \infty }{\lim }|x_{2n+2}-x_{2n+1}|=|z-P_{1}z|=\mathrm{d}(C_1,C_2),$$

$$|z_n-P_2{P}_1{z}_n{|}^2+|P_1{z}_n-P_2{P}_1{z}_n{|}^2\le |P_1{z}_n-z_n{|}^2.$$

$$\begin{array}{rl}|x_{2n+2}-z_n|& \le |P_2{x}_{2n+1}-P_2{P}_1{z}_n|+|P_2{P}_1{z}_n-z_n|\\ & \le |x_{2n+1}-P_1{z}_n|+|P_2{P}_1{z}_n-z_n|\\ & \le |x_{2n}-z_n|+|P_2{P}_1{z}_n-z_n|.\end{array}$$

$$\big{|}|x_{2n+2}-z_{n}|-|x_{2n}-z_{n}|\big{|}\leq|x_{2n+2}-x_{2n}|$$

$$\lim_{n\rightarrow\infty}\big{[}|x_{2n+1}-P_{1}z_{n}|-|P_{2}x_{2n+1}-P_{2}P_{1}z_{n}|\big{]}=0.$$

$$\lim_{n\rightarrow\infty}\big{[}x_{2n+1}-P_{1}z_{n}-(x_{2n+2}-P_{2}P_{1}z_{n})\big{]}=0,$$

$$\lim_{n\rightarrow\infty}\big{[}x_{2n+1}-P_{1}z_{n}-(x_{2n+2}-z_{n})\big{]}=0$$

$$\lim_{n\rightarrow\infty}\big{[}(x_{2n+1}-x_{2n+2})-(P_{1}z_{n}-z_{n})\big{]}=0.$$

$$\underset{n\to \infty }{\lim }|x_{2n+2}-x_{2n+1}|=\mathrm{d}(C_1,C_2),$$

Full text

2010 Mathematics Subject Classification: 41A65, 46C05, 47H09, 90C25.

Key words and phrases: Alternating nearest point projections, asymptotic behavior, asymptotically regular mapping, closed and convex set, Hilbert space, strongly nonexpansive mapping.

1 Introduction

The purpose of this note is to provide a direct proof of a result regarding the asymptotic behavior of alternating nearest point projections onto two closed and convex subsets of a Hilbert space (see Theorem 1.4 below). Our arguments are based on nonexpansive mapping theory.

Let $S_{1}$ and $S_{2}$ be two closed subspaces of a real Hilbert space $(H,\langle\cdot,\cdot\rangle)$ with induced norm $|\cdot|$, and let $P_{1}:H\to S_{1}$ and $P_{2}:H\to S_{2}$ be the corresponding orthogonal projections of $H$ onto $S_{1}$ and $S_{2}$, respectively. Denote by $\mathbb{N}=\{0,1,2,\dots\}$ the set of nonnegative integers. Let $x_{0}$ be an arbitrary point in $H$, and define the sequence $\{x_{n}:n\in\mathbb{N}\}$ of alternating projections by

[TABLE]

where $n\in\mathbb{N}$.

We begin by recalling von Neumann’s classical theorem [Ne49, page 475].

Theorem 1.1**.**

The sequence $\{x_{n}:n\in\mathbb{N}\}$ defined by (1.1) converges in norm as $n\rightarrow\infty$ to $P_{S}x_{0}$, where $P_{S}:H\to S$ is the orthogonal projection of $H$ onto the intersection $S=S_{1}\cap S_{2}$.

This theorem was rediscovered by several other authors; see, for example, [Ar50], [Na53] and [Wi55]. More information regarding this theorem and its diverse applications can be found in [De01] and the references mentioned therein. Several recent proofs of Theorem 1.1 can be found, for instance, in [BMR04], [KR04], [KR10] and [KR11].

Now let $C_{1}$ and $C_{2}$ be two nonempty, closed and convex subsets of $(H,\langle\cdot,\cdot\rangle)$, and let $P_{1}:H\to C_{1}$ and $P_{2}:H\to C_{2}$ be the corresponding nearest point projections of $H$ onto $C_{1}$ and $C_{2}$, respectively.

Let $D$ be a subset of $H$. Recall that a mapping $T:D\to H$ is called nonexpansive (that is, $1$-Lipschitz) if $|Tx-Ty|\leq|x-y|$ for all $x$ and $y$ in $D$.

Theorem 1.2**.**

Assume that the intersection $C=C_{1}\cap C_{2}$ is not empty. Then the sequence $\{x_{n}:n\in\mathbb{N}\}$ defined by (1.1) converges weakly as $n\rightarrow\infty$ to $R_{C}x_{0}$, where $R_{C}:H\to C$ is a nonexpansive retraction of $H$ onto $C$.

This theorem is due to Bregman [Br65, page 688]. It is now known [Hu04] that the sequence $\{x_{n}:n\in\mathbb{N}\}$ of alternating nearest point projections does not, in general, converge in norm. In this connection, see also [MR03] and [Ko08].

When can we be sure that norm convergence does occur and what happens when the sets $C_{1}$ and $C_{2}$ are disjoint? In order to recall certain answers to these natural questions, we first denote by ${\rm d}(C_{1},C_{2})$ the distance between the sets $C_{1}$ and $C_{2}$, that is,

[TABLE]

Now we can quote [KR04, Theorem 4.1]. In this connection, see also [BBR78], [BB93] and [BB94].

Theorem 1.3**.**

Let $C_{1}$ and $C_{2}$ be two nonempty, closed and convex subsets of the Hilbert space $H$, and let $P_{1}:H\to C_{1}$ and $P_{2}:H\to C_{2}$ be the corresponding nearest point projections of $H$ onto $C_{1}$ and $C_{2}$, respectively. Let the sequence $\{x_{n}:n\in\mathbb{N}\}$ be defined by (1.1).

1. (a)

If ${\rm d}(C_{1},C_{2})$ is attained, then the sequence $\{x_{2n}:n\in\mathbb{N}\}$ converges weakly as $n\rightarrow\infty$ to a fixed point z of $P_{2}P_{1}$ and $\{x_{2n+1}:n\in\mathbb{N}\}$ converges weakly as $n\rightarrow\infty$ to $P_{1}z$. 2. (b)

If ${\rm d}(C_{1},C_{2})$ is not attained, then the sequence $\{|x_{n}|:n\in\mathbb{N}\}\rightarrow\infty$ as $n\rightarrow\infty$. 3. (c)

If both $C_{1}$ and $C_{2}$ are symmetric with respect to the origin, then the sequence $\{x_{n}:n\in\mathbb{N}\}$ converges in norm as $n\rightarrow\infty$ to a point in the intersection $C=C_{1}\cap C_{2}$.

Part (c) of Theorem 1.3 seems to be a good nonlinear analogue of von Neumann’s linear Theorem 1.1.

Now we are ready to state the result which is of concern to us in the present note.

Theorem 1.4**.**

Let $C_{1}$ and $C_{2}$ be two nonempty, closed and convex subsets of a real Hilbert space $(H,\langle\cdot,\cdot\rangle)$ with induced norm $|\cdot|$, and let $P_{1}:H\to C_{1}$ and $P_{2}:H\to C_{2}$ be the corresponding nearest point projections of $H$ onto $C_{1}$ and $C_{2}$, respectively. Let the sequence $\{x_{n}:n\in\mathbb{N}\}$ be defined by (1.1). Then

[TABLE]

This theorem was obtained in [BB94, page 433] as a consequence of the authors’ analysis of Dykstra’s algorithm. As we have already mentioned, our goal is to provide a direct proof which is based on nonexpansive mapping theory. This proof is given in Section 3. In the next section we present several lemmata which are used in our proof.

We remark in passing that since, by the definition of the nearest point projection, we have

[TABLE]

for all $n\geq 2$, both sequences $\{|x_{2n+2}-x_{2n+1}|\}^{\infty}_{n=0}$ and $\{|x_{2n+1}-x_{2n}|\}^{\infty}_{n=1}$ decrease to their common limit.

We conclude this introduction with a corollary of Theorem 1.4. It can be proved by appealing, for example, to the parallelogram law. In this connection, see also [BB94, page 433].

Corollary 1.5**.**

In the setting of Theorem 1.4, we have

[TABLE]

where $v$ is the point of least norm in the closure of $C_{2}-C_{1}$ and the convergence is in norm.

Proof.

Let $K$ be the closure of $C_{2}-C_{1}$ and consider, for instance, the sequence $\{y_{n}:n\in\mathbb{N}\}\subset K$ defined by $y_{n}:=x_{2n+2}-x_{2n+1}$, where $n\in\mathbb{N}$. Applying the parallelogram law to $y_{n}$ and $v$, we obtain

[TABLE]

and

[TABLE]

Since $(y_{n}+v)/2\in K$, we know that $|(y_{n}+v)/2|\geq|v|$. Hence

[TABLE]

for each $n\in\mathbb{N}$. The result now follows from Theorem 1.4. ∎

2 Nearest Point Projections

Preparing for the proof of Theorem 1.4, we first present several known facts regarding nearest point projections onto closed and convex sets in Hilbert spaces.

Lemma 2.1**.**

Let $C$ be a closed and convex subset of the Hilbert space $H$, and let $P_{C}:H\to C$ be the nearest point projection of $H$ onto $C$. Then

[TABLE]

for all $x\in H$ and $y\in C$.

Proof.

We have

[TABLE]

as asserted. ∎

Let $D$ be a subset of $H$ and let $T:D\to H$ be a mapping. Recall that the mapping $T$ is said to be asymptotically regular at $x\in D$ (see, for example, [BBR78]) if it can be iterated at $x$ and $\lim_{n\rightarrow\infty}(T^{n}x-T^{n+1}x)=0$. We now quote [Ba02, Theorem 3.1, page 144]. In this connection, see also [BMMW12, Theorem 4.6, page 8].

Lemma 2.2**.**

The compositon of finitely many nearest point projections onto closed and convex subsets of a Hilbert space is asymptotically regular.

Our next lemma can be found in [CG59, page 449].

Lemma 2.3**.**

Let ${\rm d}(C_{1},C_{2})$ be the distance between two closed and convex subsets $C_{1}$ and $C_{2}$ of a Hilbert space $H$, and let $P_{1}:H\to C_{1}$ and $P_{2}:H\to C_{2}$ be the corresponding nearest point projections of $H$ onto $C_{1}$ and $C_{2}$, respectively. Then the fixed point set ${\rm F}(P_{2}P_{1})$ of the composition $P_{2}P_{1}$ in $H$ coincides with the set $\{b\in C_{2}:|b-P_{1}b|={\rm d}(C_{1},C_{2})\}$.

If $D$ is a subset of $H$, then a mapping $T:D\to H$ is said to be strongly nonexpansive [BR77] if it is nonexpansive and $\{(x_{n}-y_{n})-(Tx_{n}-Ty_{n})\}\rightarrow 0$ as $n\rightarrow\infty$ whenever $\{x_{n}\}$ and $\{y_{n}\}$ are two sequences in $D$ such that the sequence $\{x_{n}-y_{n}\}$ is bounded and $\{|x_{n}-y_{n}|-|Tx_{n}-Ty_{n}|\}\rightarrow 0$ as $n\rightarrow\infty$.

Lemma 2.4**.**

Every nearest point projection is strongly nonexpansive.

Proof.

This assertion is true because every nearest point projection in Hilbert space is known to be firmly nonexpansive (see [GR84, page 18]) and every firmly nonexpansive mapping in a uniformly convex Banach space is strongly nonexpansive by [BR77, Proposition 2.1, page 463].

∎

3 Alternating Projections

Proof of Theorem 1.4.

Since the composition $P_{2}P_{1}$ is asymptotically regular (see Lemma 2.2), the sequence $\{x_{2n+2}-x_{2n}\}\rightarrow 0$ as $n\rightarrow\infty$ and therefore the evaluation of the second limit follows from the evaluation of the first one. Alternatively, we can simply interchange the roles of $C_{1}$ and $C_{2}$.

In order to evaluate the first limit, assume initially that the distance ${\rm d}(C_{1},C_{2})$ between the closed and convex sets $C_{1}$ and $C_{2}$ is attained. Then we know (see Lemma 2.3) that

[TABLE]

Let $z\in F(P_{2}P_{1})$. Then $|z-P_{1}z|={\rm d}(C_{1},C_{2})$. Since

[TABLE]

and the sequence $\{|x_{2n}-z|\}_{n=1}^{\infty}$ is decreasing, we see that

[TABLE]

Since the projection $P_{2}$ is strongly nonexpansive (see Lemma 2.4), it follows that

[TABLE]

Hence

[TABLE]

as claimed.

Now assume that ${\rm d}(C_{1},C_{2})$ is not necessarily attained. In this case, there is, however, a sequence $\{z_{n}:n\in\mathbb{N}\}\subset C_{2}$ such that the sequence $\{|z_{n}-P_{1}{z_{n}}|\}\rightarrow{\rm d}(C_{1},C_{2})$ as $n\rightarrow\infty$.

Applying Lemma 2.1 to $C_{2}$ and $P_{2}$, we obtain, for each $n\in\mathbb{N}$,

[TABLE]

Hence $|z_{n}-P_{2}P_{1}z_{n}|^{2}+{\rm d}(C_{1},C_{2})^{2}\leq|P_{1}z_{n}-z_{n}|^{2}$ for each $n\in\mathbb{N}$. Consequently, the sequence $\{|z_{n}-P_{2}P_{1}z_{n}|\}\rightarrow 0$ as $n\rightarrow\infty$. We also have

[TABLE]

Observing that

[TABLE]

and that $\lim_{n\rightarrow\infty}|x_{2n+2}-x_{2n}|=0$ (because the composition $P_{2}P_{1}$ is asymptotically regular by Lemma 2.2), we obtain

[TABLE]

At this point we invoke Lemma 2.4 once more to obtain that

[TABLE]

[TABLE]

and

[TABLE]

Hence

[TABLE]

as asserted. ∎

Acknowledgements.

The first author was supported by Grants FWF P23628-N18, GAČR P201/12/0290 and by RVO 67985840. The second author was partially supported by the Israel Science Foundation (Grant 389/12), the Fund for the Promotion of Research at the Technion (Grant 2001893), and by the Technion General Research Fund (Grant 2016723).