On the behaviour of the Douglas-Rachford algorithm for minimizing a convex function subject to a linear constraint

TL;DR

This paper investigates the Douglas-Rachford algorithm's behavior when minimizing a convex function under a linear constraint, including cases with no common feasible points, and introduces new convergence results and parallel splitting methods.

Contribution

It provides new convergence results for the DRA in the absence of feasible points and introduces a novel parallel splitting approach for constrained convex minimization.

Findings

DRA converges to a best approximation solution even without feasible points.

New parallel splitting method for convex optimization with linear constraints.

Illustrative examples demonstrating theoretical results.

Abstract

The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the domain of the objective function and the linear subspace constraint have no point in common. As an important application, a new parallel splitting result is provided. We also illustrate our results through various examples.

Equations

$$X$$

$$U$$

$$g:X\to \left]-\infty ,+\infty \right]$$

$$\underset{\begin{array}{c}x\in X\end{array}}{\mathrm{minimize}}{\iota }_U(x)+g(x),$$

$$(T^n{x}_0{)}_{n\in \mathbb{N}}$$

$$T=\mathrm{Id}-P_U+P_g{R}_U$$

$$(P_U{T}^n{x}_0{)}_{n\in \mathbb{N}}$$

$$v=P_{\overline{\mathrm{ran}}(\mathrm{Id}-T)}(0).$$

$$Z=\big{\{}{x\in X}~{}\big{|}~{}{v\in{\operatorname{N}}_{U}(x)+\partial g(x-v)}\big{\}}\neq\varnothing.$$

$$P_Z\text{ is weak-to-weak continuous,}$$

$$0\in U^{\perp }+\mathrm{dom}{g}^{\ast },$$

$$P_U{T}^n{x}_0\rightharpoonup \text{some minimizer of ιU+g(⋅−v).}$$

$$F:=\operatorname{Fix}T(\cdot+v)=\big{\{}{x\in X}~{}\big{|}~{}{x=T(x+v)}\big{\}}\text{~{}is convex, closed, and nonempty.}$$

$$(\forall n\in \mathbb{N})T^{n}y=y-nv;$$

$$(\forall n\in \mathbb{N})\Vert (n+1)v+T^{n+1}x-y\Vert \le \Vert nv+T^{n}x-y\Vert ;$$

$$\sum _{n=0}^{+\infty }\Vert T^{n+1}x-T^{n}x-v{\Vert }^2<+\infty ,$$

$$T^{n}x-T^{n+1}x\to v;$$

$$\underset{n\to +\infty }{\lim }{P}_F(nv+T^{n}x)\in F$$

$$v_D:=P_{\overline{S_1}}(0),v_R:=P_{\overline{S_2}}(0),v:=P_{\overline{S_1}\cap \overline{S_2}}(0).$$

$$P_C=P_C\circ P_U$$

$$⟨c-P_C{P}_{U}x,x-P_C{P}_{U}x⟩$$

$$=⟨c-P_C{P}_{U}x,P_{U}x-P_C{P}_{U}x⟩$$

$$\le 0,$$

$$(\forall z\in X)h(z)\ge h(x)+⟨z-x,x^{\ast }⟩.$$

$$⟨y-x,x^{\ast }⟩=0.$$

$$h(x)=h(y).$$

$$(\forall z\in X)h(z)$$

$$=h(y)+⟨z-y,x^{\ast }⟩+⟨y-x,x^{\ast }⟩$$

$$=h(y)+⟨z-y,x^{\ast }⟩.$$

$$\mathrm{argmin}({\iota }_U+h)\rightrightarrows X:x\mapsto U^{\perp }\cap \partial h(x)$$

$$v=P_{\overline{U-\mathrm{dom}g}}(0).$$

Full text

2010 Mathematics Subject Classification: 49M27, 65K10, 90C25; Secondary 47H14, 49M29.

Keywords: convex optimization problem, Douglas-Rachford splitting, inconsistent constrained optimization, least squares solution, normal problem, parallel splitting method, projection operator, proximal mapping.

1 Introduction

Throughout, we assume that

[TABLE]

with inner product $\left\langle{\cdot},{\cdot}\right\rangle\colon X\times X\to\mathbb{R}$ and induced norm $\|\cdot\|$. We furthermore assume that

[TABLE]

and that

[TABLE]

Our aim is to discuss the behaviour of the Douglas–Rachford algorithm [17] applied to solving the optimization problem111 Let us point out that if $\widetilde{U}=\widetilde{u}+U$ is an affine subspace and $\widetilde{g}$ is convex, lower semicontinuous, and proper, then all our results are applicable by working with $U$ and $g=\widetilde{g}(\cdot-\widetilde{u})$ instead.

[TABLE]

where $\iota_{U}(x)=0$ if $x\in U$ and $\iota_{U}(x)=+\infty$ if $x\notin U$. Note that we do not assume a priori that 4 has a solution. Given any starting point $x_{0}\in X$, the Douglas–Rachford algorithm generates the so-called governing sequence

[TABLE]

where

[TABLE]

is the Douglas–Rachford operator, ${\operatorname{P}}_{U}$ is the projector of $U$, ${\operatorname{P}}_{g}$ is the proximal mapping of the function $g$, and ${\operatorname{R}}_{U}=2{\operatorname{P}}_{U}-\operatorname{Id}={\operatorname{P}}_{U}-{\operatorname{P}}_{U^{\perp}}$ is the reflector of $U$. The basic convergence result (see [22], [18], and [27]), guarantees that the shadow sequence

[TABLE]

converges weakly to a solution of (4) provided that $({\operatorname{N}}_{U}+\partial g)^{-1}(0)\neq\varnothing$.

To deal with the potential lack of solutions of (4), we define the minimal displacement vector

[TABLE]

This vector is well defined because $\overline{\operatorname{ran}}\,(\operatorname{Id}-T)$ is convex, closed, and trivially nonempty. We now assume that the so-called normal problem corresponding to (4), which asks to find a zero of the operator $-v+{\operatorname{N}}_{U}+\partial g(\cdot-v)$, admits at least one normal solution222 Note that it is possible that $Z$ is empty: indeed, consider the case when $X=\mathbb{R}=U$ and $g=\exp$. In this case, $|T^{n}x|\to+\infty$ for every $x\in\mathbb{R}$. (see [9, Definition 3.7]):

[TABLE]

We also assume throughout that

[TABLE]

which is automatically the case when $X$ is finite-dimensional, and that

[TABLE]

which is a rather mild constraint qualification that is satisfied, for instance, if $g$ has minimizers333 Also note that (11) implies that the Fenchel dual of (4) is feasible and hence that (4) is implicitly assumed to be bounded below.. Note that if (4) has a solution and $\partial(\iota_{U}+g)={\operatorname{N}}_{U}+\partial g$ (this sum formula is typically guaranteed through a regularity condition), then $v=0$ and $Z=\operatorname{argmin}(\iota_{U}+g)$. Our main result (see Theorem 5.1 below) can now be concisely stated as follows: Under the above assumptions, which we assume for the rest of the paper, we have

[TABLE]

This is a completely new (and very beautiful) variant of the classical result which is proven with a careful function value analysis in Section 4! It reveals the Douglas–Rachford algorithm to be a method for solving the following bilevel optimization problem: first, obtain the gap vector between $U=\operatorname{dom}\iota_{U}$ and $\operatorname{dom}g$. This level is purely geometrical, depending on the sets $U$ and $\operatorname{dom}g$, and revealing the minimal displacement vector $v$. Secondly, if $v\neq 0$, rather than minimizing the original $\iota_{U}+g$ which would have the optimal value $+\infty$, we then instead minimize the minimal perturbation function $\iota_{U}+g(\cdot-v)$. This has consequences for minimizing the sum of convex function by using a product space technique; in fact, real world applications inspired this research (see the last section).

Let us now comment on related previous works which will illustrate the complementary nature of the present work. To the best of our knowledge, none of these works contains the result (12) in the generality of the setting of Theorem 5.1. The paper [2] by Banjac, Goulart, Stellato, and Boyd applies the Douglas–Rachford algorithm with the function $f$ being the sum of a quadratic function and the indicator function of an affine subspace rather than $\iota_{U}$ and with $g$ being the indicator function of a nonempty closed convex set. The Douglas–Rachford method (equivalent to ADMM in this setting) is shown to be useful in providing certificates of infeasibility. The paper [8] concerns the more restrictive case when $g$ is the indicator function of a nonempty closed convex set; however, the underlying assumptions there do not require (10). The paper [9] introduces the normal problem but it does not contain any algorithmic/dynamic results. Similarly to [8], the paper [12] deals with the case when $g$ is assumed to be an indicator function of a closed affine subspace. Under suitable assumptions, the shadow sequence $({\operatorname{P}}_{U}T^{n}x_{0})_{n\in{\mathbb{N}}}$ is shown to converge strongly. The paper [13] considers an infinite-dimensional setting that encompasses two indicator functions; however, our present main result is not covered by these results (see Remark 5.4 below). In the paper [23] by Liu, Ryu, and Yin, the authors study the behaviour of the Douglas–Rachford algorithm applied to conic programming where $g$ is the indicator function of a nonempty closed convex cone while $\iota_{U}$ is replaced by the sum of a linear function and the indicator function of an affine subspace. The Douglas–Rachford method is shown to reveal information on the type of pathologies the conic program may exhibit. Finally, the paper [26] by Ryu, Liu, and Yin is the first to provide a comprehensive function-value analysis in pathological cases. It differs from the present work in that Ryu et al. allow for a general function $f$ rather than the indicator function $\iota_{U}$ considered here. However, our main result Theorem 5.1 gives information on the iterates and the function values that are not covered by the results in [26] when strong duality fails.

The remainder of this paper is organized as follows. In Section 2 we review known facts and present new auxiliary results that are needed in the main analysis. Section 3 presents new descriptions of the minimal displacement vector and the set of minimizers which are crucial in the convergence proofs. The building blocks of our analysis and the main result are presented in Sections 4 and 5 respectively. In the final Section 6, we provide a useful application of our theory to describe the behaviour of a parallel splitting method.

We employ standard notation from convex analysis and optimization as can be found, e.g., in [6] and [25].

2 Known and new auxiliary results

Because $Z\neq\varnothing$ (see 9), the generalized fixed point set introduced in [9] is very well behaved in the sense that

[TABLE]

The Douglas–Rachford operator $T$ defined in (6) enjoys the following nice properties which also underline the importance of $F$ for understanding the Douglas–Rachford algorithm:

Fact 2.1**.**

Let $x\in X$ and $y\in F$. Then444We point out that 2.1 holds in the more general setting when $T$ is any firmly nonexpansive mapping.

[TABLE]

the sequence $(nv+T^{n}x)_{n\in{\mathbb{N}}}$ is Fejér monotone with respect to $F$, i.e.,

[TABLE]

[TABLE]

[TABLE]

and the limit

[TABLE]

exists.

Proof. See [13, Corollary 4.2], [12, Proposition 2.5(vi)] and [6, Proposition 5.7]. $\hfill\quad\blacksquare$

Before we proceed, we recall the following useful fact that will be used in the proofs of Proposition 2.3 and Proposition 3.1.

Fact 2.2**.**

Let $C$ be a nonempty closed convex subset of $X$. Set $w={\operatorname{P}}_{\overline{U-C}}(0)$ and let $x\in X$. Then $w=\lim_{n\to\infty}({\operatorname{P}}_{U}-\operatorname{Id})({\operatorname{P}}_{C}{\operatorname{P}}_{U})^{n}x\in\overline{\operatorname{ran}}\,({\operatorname{P}}_{U}-\operatorname{Id})=-U^{\perp}=U^{\perp}$.

Proof. See [3, Corollary 4.6]. $\hfill\quad\blacksquare$

The next result will also be used in the proof of Proposition 3.1.

Proposition 2.3**.**

Let $C_{1}$ and $C_{2}$ be nonempty closed convex subsets of $X$, and set $S_{1}\coloneqq U-C_{1}$ and $S_{2}\coloneqq U^{\perp}-C_{2}$. Define

[TABLE]

Then the following hold:

1. (i)

$(v_{D},v_{R})\in U^{\perp}\times U$. 2. (ii)

${\operatorname{P}}_{U^{\perp}}(\overline{S_{1}})\subseteq\overline{S_{1}}$. 3. (iii)

${\operatorname{P}}_{U}(\overline{S_{2}})\subseteq\overline{S_{2}}$. 4. (iv)

$v_{D}+v_{R}\in\overline{S_{1}}\cap\overline{S_{2}}$. 5. (v)

$v=v_{D}+v_{R}$.

Proof. (i): Apply 2.2 with $(C,w)$ replaced by $(C_{1},v_{D})$ (respectively $(C,w)$ replaced by $(C_{2},v_{R})$). (ii): Let $y\in\overline{S_{1}}$. Then there exist $(u_{n})_{n\in{\mathbb{N}}}$ in $U$ and $(c_{1,n})_{n\in{\mathbb{N}}}$ is $C_{1}$ such that $u_{n}-c_{1,n}\to y$. Now, ${\operatorname{P}}_{U^{\perp}}y\leftarrow{\operatorname{P}}_{U^{\perp}}(u_{n}-c_{1,n})=-{\operatorname{P}}_{U^{\perp}}c_{1,n}={\operatorname{P}}_{U}c_{1,n}-c_{1,n}\in U-C_{1}$. Hence, ${\operatorname{P}}_{U^{\perp}}y\in\overline{U-C_{1}}=\overline{S_{1}}$ and the claim follows. (iii): Proceed similar to the proof of (ii). (iv): Indeed, note that by (i) we have $v_{R}\in U$, hence $v_{D}+v_{R}\in\overline{S_{1}}+v_{R}=\overline{U-C_{1}}+v_{R}=\overline{U-C_{1}+v_{R}}=\overline{U-C_{1}}=\overline{S_{1}}$. Similarly, we show that $v_{D}+v_{R}\in\overline{S_{2}}$ and the conclusion follows. (v): Note that (ii) & (iii) imply that $({\operatorname{P}}_{U}v,{\operatorname{P}}_{U^{\perp}}v)\in\overline{S_{2}}\times\overline{S_{1}}$. Consequently, $\lVert v_{R}\rVert\leq\lVert{\operatorname{P}}_{U}v\rVert$ and $\lVert v_{D}\rVert\leq\lVert{\operatorname{P}}_{U^{\perp}}v\rVert$. Altogether, in view of (i), we learn that $\lVert v_{D}+v_{R}\rVert^{2}=\lVert v_{D}\rVert^{2}+\lVert v_{R}\rVert^{2}\leq\lVert{\operatorname{P}}_{U}v\rVert^{2}+\lVert{\operatorname{P}}_{U^{\perp}}v\rVert^{2}=\lVert v\rVert^{2}$. Combining this with (iv), and the definition of $v$, we obtain the result. $\hfill\quad\blacksquare$

The following simple result, which relies on the assumption that $U$ is a closed linear subspace, will be used in the proof of Theorem 5.1.

Lemma 2.4**.**

Let $C$ be a nonempty closed convex subset of $U$. Then

[TABLE]

Proof. Let $x\in X$ and let $c\in C\subseteq U$. Then ${\operatorname{P}}_{C}{\operatorname{P}}_{U}x\in C$ and

[TABLE]

and we are done. $\hfill\quad\blacksquare$

We now turn to the minimization of a convex function subject to a linear constraint. The following result will be used in the proof of Theorem 3.4.

Lemma 2.5**.**

Let $h\colon X\to\,\left]-\infty,+\infty\right]$ be a proper lower semicontinuous convex function. Furthermore, let $x$ and $y$ be points in $U$, and let $x^{*}\in X$. Then the following hold:

1. (i)

If $U^{\perp}\cap\partial h(x)\neq\varnothing$, then $x$ is a minimizer of $\iota_{U}+h$. 2. (ii)

If $x^{*}\in U^{\perp}\cap\partial h(x)$ and $y$ is a minimizer of $\iota_{U}+h$, then $x^{*}\in U^{\perp}\cap\partial h(y)$.

Proof. (i): Suppose that $U^{\perp}\cap\partial h(x)\neq\varnothing$. Then, since $U^{\perp}$ is a subspace, $(-U^{\perp})\cap\partial h(x)\neq\varnothing$. Suppose that $x^{*}\in\partial h(x)$. Then $-x^{*}\in U^{\perp}={\operatorname{N}}_{U}(x)$. It follows that $0=(-x^{*})+x^{*}\in{\operatorname{N}}_{U}(x)+\partial h(x)=\partial\iota_{U}(x)+\partial h(x)\subseteq\partial(\iota_{U}+h)(x)$. By Fermat’s rule, $x$ is a minimizer of $\iota_{U}+h$.

(ii): Suppose that $x^{*}\in U^{\perp}\cap\partial h(x)\neq\varnothing$. Then

[TABLE]

and

[TABLE]

On the other hand, because $y$ is a minimizer of $\iota_{U}+h$, we learn from (i) that

[TABLE]

Altogether,

[TABLE]

Therefore, $x^{*}\in\partial h(y)$. $\hfill\quad\blacksquare$

The assumption that $U^{\perp}\cap\partial h(x)\neq\varnothing$ in Lemma 2.5(ii) is critical:

Example 2.6**.**

Suppose that $X=\mathbb{R}$, that $U=\{0\}$, and that $h(\xi)=-\sqrt{\xi}$, if $\xi\geq 0$ and $h(\xi)=+\infty$ if $\xi<0$. Then [math] minimizes $\iota_{U}+h=\iota_{U}$ yet $U^{\perp}\cap\partial h(0)=\partial h(0)=\varnothing$.

Remark 2.7**.**

Let $h\colon X\to\,\left]-\infty,+\infty\right]$ be a proper lower semicontinuous convex function. Then Lemma 2.5 implies that the set-valued operator

[TABLE]

is constant.

3 New static results

We start with the following useful result for the minimal displacement vector $v$ from (8).

Proposition 3.1**.**

Set $w={\operatorname{P}}_{\overline{U-{\operatorname{dom}g}}}(0)$. Then the following hold:

1. (i)

$w\in U^{\perp}$. 2. (ii)

If $X$ is finite-dimensional, then $v=w={\operatorname{P}}_{\overline{U-\operatorname{dom}g}}(0)\in U^{\perp}$.

Proof. Clearly $\overline{U-\operatorname{dom}g}=\overline{U-\overline{\operatorname{dom}}g}$ and, $\overline{U^{\perp}+\operatorname{dom}g^{*}}=\overline{U^{\perp}+\overline{\operatorname{dom}}g^{*}}$. (i): Apply 2.2 with $C$ replaced by $\overline{\operatorname{dom}}g$. (ii): Note that $\iota_{U}^{*}=\iota_{U^{\perp}}$ and thus $\operatorname{dom}\iota_{U}^{*}=U^{\perp}$. Hence (11) states exactly that $0\in\operatorname{dom}\iota_{U}^{*}+\operatorname{dom}g^{*}$. It follows from [10, Proposition 6.1(ii) and Corollary 6.5(i)] that $v={\operatorname{P}}_{\overline{(U-\operatorname{dom}g)}\cap\overline{(U^{\perp}+\operatorname{dom}g^{*})}}(0)$. By Proposition 2.3 applied with $(C_{1},C_{2})$ replaced by $(\overline{\operatorname{dom}g},-\overline{\operatorname{dom}}g^{*})$ we have

[TABLE]

Now combine with (i). $\hfill\quad\blacksquare$

The result in Proposition 3.1(ii) was first proved — in an even more general form — by Ryu, Liu, and Yin with a different argument relying on recession functions (see [26, Lemma 3]). From now on, we assume:

[TABLE]

Note that 28 holds if $X$ is finite-dimensional by Proposition 3.1(ii). In view of Proposition 3.1(i), we have

[TABLE]

The fact that $v$ belongs to $U^{\perp}$ is new and crucial to our analysis.

We now turn towards alternative descriptions of the set $Z$ of normal solutions, defined in (9). In passing, we mention that the next result is true even if $Z=\varnothing$.

Proposition 3.2**.**

We have

[TABLE]

and

[TABLE]

Proof. Recall that $v\in U^{\perp}$ by (29). Hence ${\operatorname{N}}_{U}=-v+{\operatorname{N}}_{U}$. Now let $x\in X$. Then

[TABLE]

which proves 30, 31a, and 31b. Turning to 31c, let $x\in\operatorname{zer}({\operatorname{N}}_{U}+\partial g(\cdot-v))$. On the one hand, $x\in\operatorname{dom}({\operatorname{N}}_{U}+\partial g(\cdot-v))$ and thus ${\operatorname{N}}_{U}(x)\neq\varnothing$ and $\partial g(x-v)\neq\varnothing$. Hence $x\in U$ and $x-v\in\operatorname{dom}\partial g$, i.e., $x\in U\cap(v+\operatorname{dom}\partial g$. On the other hand, $\operatorname{zer}({\operatorname{N}}_{U}+\partial g(\cdot-v))=\operatorname{zer}(\partial\iota_{U}+\partial g(\cdot-v))$. Hence $0\in\partial\iota_{U}(x)+\partial g(\cdot-v)(x)\subseteq\partial(\iota_{U}+g(\cdot-v))(x)$ and therefore $x$ minimizes $\iota_{U}+g(\cdot-v)$. Finally, 31d and 31e are obvious. $\hfill\quad\blacksquare$

Example 3.3** **(linear-convex feasibility).

Suppose that $g=\iota_{W}$, where $W$ is a nonempty closed convex subset of $X$. Then $v={\operatorname{P}}_{\overline{U-W}}(0)$, $\operatorname{argmin}g=\operatorname{dom}\partial g=W$, and $v+\operatorname{argmin}g=v+W=v+\operatorname{dom}g$. Thus Proposition 3.2 yields

[TABLE]

a result that is well known (see [7]).

We are now ready for our first main result which provides a useful description of $Z$:

Theorem 3.4**.**

Because $Z$ is nonempty, we have

[TABLE]

Proof. Proposition 3.2 yields the inclusions Z\subseteq U\cap(v+\operatorname{dom}\partial g)\cap\\ \ \big{(}\iota_{U}+g(\cdot-v)\big{)}\subseteq\operatorname{argmin}\big{(}\iota_{U}+g(\cdot-v)\big{)}. Because $Z\neq\varnothing$, we let $x\in Z$, and also let $y\in\operatorname{argmin}(\iota_{U}+g(\cdot-v))\subseteq U$. First, by (30), $x\in U$ and $U^{\perp}\cap\partial g(x-v)\neq\varnothing$. Secondly, it follows from Lemma 2.5 (applied with $h=g(\cdot-v)$) that $U^{\perp}\cap\partial g(y-v)\neq\varnothing$. Therefore, by using again 30, we obtain $y\in Z$. $\hfill\quad\blacksquare$

Here is an example of a case where $Z\neq\varnothing$.

Example 3.5**.**

Suppose that $g$ is polyhedral. Then [4, Theorem 5.6.1] implies that $U\cap(v+\operatorname{dom}g)=U\cap\operatorname{dom}g(\cdot-v)\neq\varnothing$. Hence, by [6, Corollary 27.3(c)] we have Z=\operatorname{argmin}\big{(}\iota_{U}+g(\cdot-v)\big{)}.

The underlying assumption that $Z$ be nonempty (see 9) in Theorem 3.4 is critical:

Example 3.6**.**

Suppose that $X=\mathbb{R}^{2}$, that $U=\{0\}\times\mathbb{R}$ and that $g$ is the Rockafellar function defined by

[TABLE]

(see [25, Example on page 218]). Then $v=0$ and it follows from [24, Example 7.5] that $Z=\varnothing$, $\operatorname{argmin}(\iota_{U}+g(\cdot-v))=\{0\}\times[-1,1]$, and $U\cap(v+\operatorname{dom}\partial g)\cap\operatorname{argmin}(\iota_{U}+g(\cdot-v))=\{0\}\times\{-1,1\}$.

Proof. Clearly we have $U^{\perp}=\mathbb{R}\times\{0\}$ and $\operatorname{dom}g=\mathbb{R}_{+}\times\mathbb{R}$. Moreover, [24, Example 6.5] implies that \operatorname{dom}\partial g=\big{\{}{(\xi_{1},\xi_{2})}~{}\big{|}~{}{\xi_{1}>0,\xi_{2}\in\mathbb{R}}\big{\}}\cup\big{\{}{(0,\xi_{2})}~{}\big{|}~{}{\xi_{2}\geq 1}\big{\}}, and \operatorname{dom}\partial g^{*}=\operatorname{dom}g^{*}=\big{\{}{(\xi_{1},\xi_{2})}~{}\big{|}~{}{\xi_{1}\leq 0,\lvert\xi_{2}\rvert\leq 1}\big{\}}. Therefore, using [10, Corollary 6.5(i)] we learn that $v={\operatorname{P}}_{(\overline{U-\operatorname{dom}}g)\cap(\overline{U^{\perp}+\operatorname{dom}}g^{*})}(0)=0$. It follows from Proposition 3.2 that Z=\big{\{}{(0,\xi_{2})}~{}\big{|}~{}{U^{\perp}\cap\partial g((0,\xi_{2}))\neq\varnothing}\big{\}}. Now let $(0,\xi_{2})\in U\cap\operatorname{dom}g$ and note that [24, Example 6.5] implies that

[TABLE]

which proves the claim that $Z=\varnothing$. Finally, using 35, we see that $\operatorname{argmin}(\iota_{U}+g(\cdot-v))=\operatorname{argmin}(\iota_{U}+g)=\{0\}\times[-1,1]$ and the conclusion follows. $\hfill\quad\blacksquare$

When $X=\mathbb{R}$, then we obtain the following positive result, which holds even when $Z=\varnothing$:

Proposition 3.7**.**

Suppose that $X=\mathbb{R}$. Then

[TABLE]

More precisely, exactly one of the following cases holds:

1. (i)

$U=\{0\}$, $v={\operatorname{P}}_{-\overline{\operatorname{dom}}\,g}(0)$, $Z=0\cdot\partial g(-v)$, and either $\iota_{U}+g(\cdot-v)=\iota_{\{0\}}$ if $-v\in\operatorname{dom}g$ or $\iota_{U}+g(\cdot-v)=\iota_{\varnothing}$ if $-v\notin\operatorname{dom}g$. 2. (ii)

$U=\mathbb{R}$, $v=0$, and $Z=\operatorname{dom}\partial g\cap\operatorname{argmin}g=\operatorname{argmin}g$.

Proof. Denote the right side of 37 by $R$. It is clear from Proposition 3.2 that $Z\subseteq R$. Now let $x\in R$. On the one hand,

[TABLE]

On the other hand, $x\in\operatorname{dom}\partial\iota_{U}\cap\operatorname{dom}\partial g(\cdot-v)$. By the sum rule for the real line, we have

[TABLE]

Altogether, $0\in\partial\iota_{U}(x)+\partial g(x-v)$ and thus $x\in Z$ by Proposition 3.2. The remaining statements follow readily. $\hfill\quad\blacksquare$

The previous results make it tempting to conjecture that when $X=\mathbb{R}$ and $Z=\varnothing$, then we have $\operatorname{argmin}(\iota_{U}+g(\cdot-v))=\varnothing$. Unfortunately, this conjecture is false:

Example 3.8**.**

Suppose that $X=\mathbb{R}$, that $U=\{0\}$ and that $-\sqrt{x}$ with $\operatorname{dom}g=\mathbb{R}_{+}$. Then $v={\operatorname{P}}_{-\overline{\operatorname{dom}}\,g}(0)=0$. Hence $Z=\{0\}\cdot\partial g(0)=\varnothing$ by Proposition 3.7 while $\operatorname{argmin}(\iota_{U}+g(\cdot-v))=\{0\}$ because $\iota_{U}+g(\cdot-v)=\iota_{U}+g=\iota_{U}=\iota_{\{0\}}$.

We conclude this section with another useful consequence of (29):

Proposition 3.9**.**

We have $Z={\operatorname{P}}_{U}(F)$ and

[TABLE]

Proof. Set $A=-v+{\operatorname{N}}_{U}$ and $B=\partial g(\cdot-v)$, and note that by (29) $A={\operatorname{N}}_{U}$. Then the Douglas–Rachford operator corresponding to $(A,B)$ is [9, Proposition 3.2]

[TABLE]

Moreover ${\operatorname{J}}_{A}:=(\operatorname{Id}+A)^{-1}={\operatorname{P}}_{U}$. Note that $A$ and $B$ are subdifferential operators, hence paramonotone by [19, Theorem 2.2]. So [5, Corollary 5.6] yields $F=Z+K$, $Z={\operatorname{J}}_{A}(F)={\operatorname{P}}_{U}(F)$, where $K:=(\operatorname{Id}-{\operatorname{J}}_{A^{-1}})(F)={\operatorname{P}}_{U^{\perp}}(F)\subseteq U^{\perp}$. Moreover, because $Z-Z\subseteq U$ and so $Z-Z\perp K$, we have ${\operatorname{J}}_{A}{\operatorname{P}}_{Z+K}={\operatorname{P}}_{Z}$, equivalently, ${\operatorname{P}}_{U}{\operatorname{P}}_{F}={\operatorname{P}}_{Z}$, by [5, Theorem 6.7(ii)]. $\hfill\quad\blacksquare$

4 New dynamic results

Recall that

[TABLE]

We start with a result that provides some information on the shadow sequence $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$. (In passing, we note that only item (v) requires that $Z$ be nonempty.)

Lemma 4.1**.**

Let $x\in X$. Then the following hold:

1. (i)

${\operatorname{P}}_{U}T^{n}x-{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x=T^{n}x-T^{n+1}x\to v\in U^{\perp}$. 2. (ii)

${\operatorname{P}}_{U}T^{n}x-{\operatorname{P}}_{U}{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x={\operatorname{P}}_{U}T^{n}x-{\operatorname{P}}_{U}T^{n+1}x\to 0$. 3. (iii)

$-{\operatorname{P}}_{U^{\perp}}{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x={\operatorname{P}}_{U^{\perp}}T^{n}x-{\operatorname{P}}_{U^{\perp}}T^{n+1}x\to v$*.

4. (iv)

All weak cluster points of $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ lie in $U\cap(v+\overline{\operatorname{dom}}\,g)$. 5. (v)

The sequences $(nv+T^{n}x)_{n\in{\mathbb{N}}}$, $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$, and $({\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ are bounded.

Proof. (i): Clear from the definition of $T$, (17) and (29). (ii): Apply ${\operatorname{P}}_{U}$ to (i). (iii): Apply ${\operatorname{P}}_{U^{\perp}}$ to (i). (iv): On the one hand, $(T^{n}x-T^{n+1}x)+{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x={\operatorname{P}}_{U}T^{n}x\in U$. On the other hand, ${\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x\in\operatorname{dom}\partial g\subseteq\overline{\operatorname{dom}}\,g$. Altogether, combined with (i), we obtained the desired result. (v): By 2.1 and 13, the sequence $(nv+T^{n}x)_{n\in{\mathbb{N}}}$ is Fejér monotone with respect to $F\neq\varnothing$, hence it is bounded. Therefore, $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}=({\operatorname{P}}_{U}(nv+T^{n}x))_{n\in{\mathbb{N}}}$ is also bounded. The boundedness of $({\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ follows from (i). $\hfill\quad\blacksquare$

Note that Proposition 3.2 yields that $Z-v\subseteq(U-v)\cap\operatorname{dom}g$, and thus $U-v\cap\operatorname{dom}g$ is nonempty. The next result provides information on function values of $g$ of a sequence occurring in the Douglas–Rachford algorithm.

Lemma 4.2**.**

Let $x\in X$, let $y\in(U-v)\cap\operatorname{dom}g$, and let ${n\in{\mathbb{N}}}$. Then

[TABLE]

Proof. The characterization of the prox operator ${\operatorname{P}}_{g}$ gives

[TABLE]

We also have

[TABLE]

Now write $y=u-v$, where $u\in U$. Then, using also the identity in Lemma 4.1(iii) to derive 46e, we have

[TABLE]

Therefore, substituting 45 and 46 into 44, we obtain

[TABLE]

which completes the proof. $\hfill\quad\blacksquare$

We are now able to locate weak cluster points of the shadow sequence $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$:

Lemma 4.3**.**

Let $x\in X$ and let $y\in(U-v)\cap\operatorname{dom}g$. Then there exists a sequence $(\varepsilon_{n})_{n\in{\mathbb{N}}}$ in $\mathbb{R}$ such that

[TABLE]

and for every ${n\in{\mathbb{N}}}$, we have

[TABLE]

Moreover, the sequence

[TABLE]

[TABLE]

and

[TABLE]

Finally, the sequence

[TABLE]

Proof. Lemma 4.1(v)&(i) yield that $(y-{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ is bounded and that ${\operatorname{P}}_{U}T^{n}x-v-{\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x\to 0$. Thus

[TABLE]

Lemma 4.1(iii)&(i) yield that ${\operatorname{P}}_{U^{\perp}}T^{n}x-{\operatorname{P}}_{U^{\perp}}T^{n+1}x-v\to 0$ and that $({\operatorname{P}}_{U^{\perp}}(nv+T^{n}x))_{n\in{\mathbb{N}}}$ is bounded. Hence

[TABLE]

Setting

[TABLE]

we see that 49 is a consequence of Lemma 4.2, 54 and 55.

By Lemma 4.1(v), $({\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ is bounded. Let $c$ be a weak cluster point of $({\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$, say ${\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{k_{n}}x\>{\rightharpoonup}\>c$. Lemma 4.1(i) implies that

[TABLE]

Now abbreviate \alpha_{n}=(n+1)\big{\langle}{T^{n}x-T^{n+1}x-v},{v}\big{\rangle}. Then 49 yields

[TABLE]

The weak lower semicontinuity of $g$ now yields

[TABLE]

Combining with 58, we deduce that

[TABLE]

Set $\mu=\inf g(U-v)$. Choosing $y=c$ in 60 yields

[TABLE]

Now choosing $y$ so that $g(y)$ is as close to $\mu$ as we like, we deduce from 60 and 62 that

[TABLE]

Hence $c$ is a minimizer of $\iota_{U-v}+g$. Because $c$ was an arbitrary weak cluster point of $({\operatorname{P}}_{g}{\operatorname{R}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$, we obtain through a simple proof by contradiction that

[TABLE]

i.e., 51 holds.

Next, 59 with $y=c$ yields $\mu=g(c)\geq\mu+\varlimsup\alpha_{n}\geq\mu+\varliminf\alpha_{n}\geq\mu$. Thus $\alpha_{n}\to 0$ and 52 follows.

Finally, 53 follows from 50 and Lemma 4.1(i). $\hfill\quad\blacksquare$

Remark 4.4**.**

Note that (52) is equivalent to $n\cdot\left\langle{T^{n}x-T^{n+1}x-v},{v}\right\rangle\to 0$. On the other hand, (15) and (16) combined with [21, Chapter III, Section 14, Theorem on p. 124] (or [20, Problem 3.2.35]) yields ${n}\cdot\|T^{n}x-T^{n+1}x-v\|^{2}\to 0$. We do not know whether $n\cdot\|T^{n}x-T^{n+1}x-v\|\to 0$.

5 The main result

We are now ready for the main result. In the following we set

[TABLE]

which is well defined by 2.1.

Theorem 5.1** **(main result).

Let $x\in X$. Then

[TABLE]

$T^{n+1}x-T^{n}x+{\operatorname{P}}_{U}T^{n}x={\operatorname{P}}_{g}({\operatorname{R}}_{U}T^{n}x)\>{\rightharpoonup}\>-v+{\operatorname{P}}_{U}y(x)$, and

[TABLE]

Proof. For brevity, we write $y=y(x)$. Because ${\operatorname{P}}_{U}$ is continuous, we have

[TABLE]

On the other hand, ${\operatorname{P}}_{U}{\operatorname{P}}_{F}={\operatorname{P}}_{Z}={\operatorname{P}}_{Z}{\operatorname{P}}_{U}$ by 40 and 20. Invoking the fact that $v\in U^{\perp}$ (see 29), we conclude altogether that

[TABLE]

Recall from (53) and (34) that $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ is bounded and that all its cluster points lie in $\operatorname{argmin}(\iota_{U}+g(\cdot-v))=Z$. Now let $z$ be an arbitrary weak cluster point of $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$, say ${\operatorname{P}}_{U}T^{k_{n}}x\>{\rightharpoonup}\>z\in Z\subseteq U$. Then ${\operatorname{P}}_{Z}{\operatorname{P}}_{U}T^{k_{n}}x\>{\rightharpoonup}\>{\operatorname{P}}_{Z}z=z$ using (10). Combining with 69, we deduce that $z={\operatorname{P}}_{U}y$. Hence every weak cluster point of $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ coincides with ${\operatorname{P}}_{U}y$. In view of the boundedness of $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$, we obtain (66). The remainder follows from Lemma 4.1(i) and (51). $\hfill\quad\blacksquare$

Example 5.2** **(linear-convex feasibility).

Suppose that $g=\iota_{W}$, where $W$ is a nonempty closed convex subset of $X$ such that $U\cap(v+W)\neq\varnothing$. Then, $0\in\operatorname{dom}g^{*}$ which implies that $0\in U^{\perp}+\operatorname{dom}g^{*}$, hence 11 is verified. Moreover, $v={\operatorname{P}}_{\overline{U-W}}(0)$ by [9, Proposition 3.16] and $(\forall x\in X)$ ${\operatorname{P}}_{U}T^{n}x\>{\rightharpoonup}\>{\operatorname{P}}_{U}y\in U\cap(v+W)$, where $y=\lim_{n\to\infty}{\operatorname{P}}_{F}(nv+T^{n}x)$ by Theorem 5.1.

Example 5.3**.**

Suppose that $W$ is a linear subspace of $X$ such that $\{0\}\subsetneqq W\subsetneqq U^{\perp}$. Let $w\in W\smallsetminus\{0\}$, let $b\in(U^{\perp}\cap W^{\perp})\smallsetminus\{0\}$, and suppose that $g=\tfrac{1}{2}\lVert\cdot\rVert^{2}+\left\langle{w},{\cdot}\right\rangle+\iota_{-b+W}$. Let $x\in X$. Then the following hold:

1. (i)

$\partial g=w+\operatorname{Id}+{\operatorname{N}}_{-b+W}$. 2. (ii)

$U\cap W=\{0\}$. 3. (iii)

$\operatorname{dom}g=\operatorname{dom}\partial g=-b+W$, $\operatorname{dom}g^{*}=X$, and $0\in U^{\perp}+\operatorname{dom}g^{*}=X$. 4. (iv)

$v=b\in U^{\perp}\cap W^{\perp}$. 5. (v)

$-v+{\operatorname{N}}_{U}={\operatorname{N}}_{U}$. 6. (vi)

$Z=\{0\}$. 7. (vii)

${\operatorname{P}}_{g}=-b-\tfrac{1}{2}w+\tfrac{1}{2}{\operatorname{P}}_{W}$*. * 8. (viii)

$T=-b-\tfrac{1}{2}w+\operatorname{Id}-{\operatorname{P}}_{U}-\tfrac{1}{2}{\operatorname{P}}_{W}$. 9. (ix)

$F=U^{\perp}\cap(-w+W^{\perp})$. 10. (x)

$0\notin F$. 11. (xi)

$(\forall n\geq 1)$* $T^{n}x=({\operatorname{P}}_{U^{\perp}}-(1-\tfrac{1}{2^{n}}){\operatorname{P}}_{W})x-nb-(1-\tfrac{1}{2^{n}})w$.* 12. (xii)

$(\forall n\geq 1)$* ${\operatorname{P}}_{U}T^{n}x=0$.*

Proof. Note that $U+W\subsetneqq U+U^{\perp}=X$ and thus $U^{\perp}\cap W^{\perp}=(U+W)^{\perp}\supsetneqq\{0\}$. Hence the choice of $b$ is possible. (i): Clear. (ii): Indeed, $\{0\}\subseteq U\cap W\subseteq U\cap U^{\perp}=\{0\}$. (iii): It is clear that $\operatorname{dom}g=\operatorname{dom}\partial g=-b+W$. Because $\lim_{\|x\|\to+\infty}g(x)/\|x\|=+\infty$, it follows that $\operatorname{dom}g^{*}=\operatorname{dom}\partial g^{*}=X$ by, e.g., [6, Proposition 14.15 and Proposition 16.27]. (iv): Using (29) and (iii), we obtain $v={\operatorname{P}}_{\overline{U-\operatorname{dom}g}}(0)={\operatorname{P}}_{b+U+W}(0)=b+{\operatorname{P}}_{U+W}(0-b)={\operatorname{P}}_{(U+W)^{\perp}}(b)={\operatorname{P}}_{U^{\perp}\cap W^{\perp}}(b)=b$. (v): Clear from (iv). (vi): This follows from (9), (i), (ii), and (iii). (vii): Set $y=-b-\tfrac{1}{2}w+\tfrac{1}{2}{\operatorname{P}}_{W}x$. Then $y\in-b+W$. Thus, ${\operatorname{P}}_{W^{\perp}}x\in-2b+W^{\perp}$ $\Leftrightarrow$ $x\in 2(-b-\tfrac{1}{2}w+\tfrac{1}{2}{\operatorname{P}}_{W}x)+w+W^{\perp}=2y+w+W^{\perp}=y+w+y+{\operatorname{N}}_{-b+W}(y)=(\operatorname{Id}+\partial g)(y)$ $\Leftrightarrow$ $y={\operatorname{P}}_{g}(x)$. (viii): This follows from (6) and (vii). (ix): Using (13) and (viii), we obtain $x\in F$ $\Leftrightarrow$ $x=T(x+v)=T(x+b)$ $\Leftrightarrow$ $x=-b-\tfrac{1}{2}w+x+b-{\operatorname{P}}_{U}(x+b)-\tfrac{1}{2}{\operatorname{P}}_{W}(x+b)$ $\Leftrightarrow$ $0=\tfrac{1}{2}w+\tfrac{1}{2}{\operatorname{P}}_{U}x+\tfrac{1}{2}{\operatorname{P}}_{W}x$ $\Leftrightarrow$ [$x\in U^{\perp}$ and $x\in-w+W^{\perp}$]. (x): We have the equivalences $0\in F$ $\Leftrightarrow$ $0=T(0+v)$ $\Leftrightarrow$ $0=T(b)$ $\Leftrightarrow$ $0=-b-\tfrac{1}{2}w+b-{\operatorname{P}}_{U}b-\tfrac{1}{2}{\operatorname{P}}_{W}b$ $\Leftrightarrow$ $0=-\tfrac{1}{2}w$, which is absurd. (xi): This follows from (ix) and induction. (xii): Clear from (xi). $\hfill\quad\blacksquare$

Remark 5.4**.**

We point out that in [13, Theorem 4.4] the authors provide an instance where the shadow sequence converges. The proof in [13] critically relies on the assumption that $Z\subseteq F$. Our new result does not require this assumption. Indeed, by Example 5.3(vi)&(x), $Z=\{0\}$ and $Z\cap F=\varnothing$.

Example 5.5**.**

Suppose that $X$ is finite-dimensional555 We require this assumption in the proof of item (v) which relies on [10]. , that $U\neq\{0\}$, let $u^{*}\in U\smallsetminus\{0\}$, suppose that666Given a nonempty closed convex subset $C$ of $X$, the associated distance function to the set $C$ is denoted by ${\operatorname{dist}}_{C}$. $g=\tfrac{1}{2}{\operatorname{dist}}_{U}^{2}+\left\langle{u^{*}},{\cdot}\right\rangle$, and let $x\in X$. Then the following hold:

1. (i)

$\partial g=\nabla g=u^{*}+{\operatorname{P}}_{U^{\perp}}$. 2. (ii)

$U-\operatorname{dom}\nabla g=U-\operatorname{dom}g=X$. 3. (iii)

${\operatorname{ran}}\,{\operatorname{N}}_{U}+{\operatorname{ran}}\,\partial g=U^{\perp}+\operatorname{dom}g^{*}=U^{\perp}+\operatorname{dom}\partial g^{*}=u^{*}+U^{\perp}$* is closed.* 4. (iv)

$0\not\in\overline{U^{\perp}+\operatorname{dom}g^{*}}=\overline{{\operatorname{ran}}\,{\operatorname{N}}_{U}+{\operatorname{ran}}\,\partial g}$. 5. (v)

$v=u^{*}\in U\smallsetminus\{0\}$. 6. (vi)

$Z=U$. 7. (vii)

${\operatorname{P}}_{g}=-u^{*}+\operatorname{Id}-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}$. 8. (viii)

$T={\operatorname{P}}_{g}=-u^{*}+\operatorname{Id}-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}$. 9. (ix)

$F=U$. 10. (x)

$(\forall{n\in{\mathbb{N}}})$* $T^{n}x=-nu^{*}+{\operatorname{P}}_{U}x+\tfrac{1}{2^{n}}{\operatorname{P}}_{U^{\perp}}x$.* 11. (xi)

$(\forall{n\in{\mathbb{N}}})$* ${\operatorname{P}}_{U}T^{n}x=-nu^{*}+{\operatorname{P}}_{U}x$.* 12. (xii)

$(\forall{n\in{\mathbb{N}}})$* $\lVert T^{n}x\rVert\geq\lVert{\operatorname{P}}_{U}T^{n}x\rVert\geq n\lVert u^{*}\rVert-\lVert{\operatorname{P}}_{U}x\rVert\to+\infty$. *

Proof. (i): Clear since $\nabla\tfrac{1}{2}{\operatorname{dist}}_{U}^{2}=\operatorname{Id}-{\operatorname{P}}_{U}={\operatorname{P}}_{U^{\perp}}$. Note that $\nabla g=u^{*}+\operatorname{Id}-{\operatorname{P}}_{U}=u^{*}+{\operatorname{P}}_{U^{\perp}}$. (ii): $U-\operatorname{dom}\partial g=U-X=X$. (iii): $\operatorname{dom}\partial g^{*}={\operatorname{ran}}\,\nabla g=u^{*}+U^{\perp}$ is closed. On the other hand, $\operatorname{dom}\partial g^{*}$ is a dense subset of $\overline{\operatorname{dom}}\,g^{*}$. Hence $\operatorname{dom}\partial g^{*}=\operatorname{dom}g^{*}=u^{*}+U^{\perp}$ and thus ${\operatorname{ran}}\,{\operatorname{N}}_{U}+{\operatorname{ran}}\,\partial g=U^{\perp}+(u^{*}+U^{\perp})=u^{*}+U^{\perp}$. (iv): Clear from (iii) and the assumption that $u^{*}\neq 0$. (v): By [10, Proposition 6.1], (ii), and (iii), we have $v={\operatorname{P}}_{\overline{U-\operatorname{dom}g}\ \cap\ \overline{U^{\perp}+\operatorname{dom}g^{*}}}(0)={\operatorname{P}}_{u^{*}+U^{\perp}}(0)=u^{*}+{\operatorname{P}}_{U^{\perp}}(0-u^{*})={\operatorname{P}}_{U}(u^{*})=u^{*}$. (vi): Using (9), (i), and (v), we have $x\in Z$ $\Leftrightarrow$ $v\in{\operatorname{N}}_{U}(x)+\partial g(x-v)$ $\Leftrightarrow$ [$x\in U$ and $u^{*}\in U^{\perp}+u^{*}+{\operatorname{P}}_{U^{\perp}}(x-u^{*})$] $\Leftrightarrow$ $x\in U$. (vii): Set $y=-u^{*}+x-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}x$. By (i) and (v), $y+\nabla g(y)=(-u^{*}+x-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}x)+(u^{*}+{\operatorname{P}}_{U^{\perp}}(-u^{*}+x-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}x))=x$. Thus $y={\operatorname{P}}_{g}(x)$ as claimed. (viii): Using (6) and (vii), we obtain $T=\operatorname{Id}-{\operatorname{P}}_{U}+{\operatorname{P}}_{g}{\operatorname{R}}_{U}={\operatorname{P}}_{U^{\perp}}+{\operatorname{P}}_{g}({\operatorname{P}}_{U}-{\operatorname{P}}_{U^{\perp}})={\operatorname{P}}_{U^{\perp}}-u^{*}+(\operatorname{Id}-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}})({\operatorname{P}}_{U}-{\operatorname{P}}_{U^{\perp}})=-u^{*}+{\operatorname{P}}_{U}+\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}=-u^{*}+{\operatorname{P}}_{U}+{\operatorname{P}}_{U^{\perp}}-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}=-u^{*}+\operatorname{Id}-\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}={\operatorname{P}}_{g}$. (ix): Using (13), (v), and (viii), we have $x\in F$ $\Leftrightarrow$ $x=T(x+v)$ $\Leftrightarrow$ $x=-u^{*}+{\operatorname{P}}_{U}x+\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}(x+v)$ $\Leftrightarrow$ $x={\operatorname{P}}_{U}x+\tfrac{1}{2}{\operatorname{P}}_{U^{\perp}}x$ $\Leftrightarrow$ $x\in U$. (x): This follows from (viii) and (v) by a straight-forward induction. (xi): Apply ${\operatorname{P}}_{U}$ to (x) and use (v). (xii): This follows from (xi). $\hfill\quad\blacksquare$

Remark 5.6**.**

Example 5.5* illustrates the importance of the constraint qualification (11); indeed, it provides a scenario where (11) fails (see item (iv)) and the shadow sequence never converges (see item (xii)).*

Remark 5.7**.**

While Theorem 5.1 guarantees that $({\operatorname{P}}_{U}T^{n}x)_{n\in{\mathbb{N}}}$ converges weakly to a minimizer of $\iota_{U}+g(\cdot-v)$, we leave numerical experiments and the development of meaningful termination criteria as topics for future research. A promising starting point appears to be the analysis in [2, Section 5].

The remaining results in this section were inspired by a referee’s question.

Theorem 5.8** **(switching the order of the operators).

*Set $\widetilde{T}=\operatorname{Id}-{\operatorname{P}}_{g}+{\operatorname{P}}_{U}{\operatorname{R}}_{g}=\operatorname{Id}-{\operatorname{P}}_{g}+{\operatorname{P}}_{U}(2{\operatorname{P}}_{g}-\operatorname{Id})$. Suppose that777This assumption is satisfied if, for instance, $X$ is finite-dimensional. To see this, proceed as in the proof of Proposition 3.1(ii), with the roles of $\iota_{U}$ and $g$ switched.

${\operatorname{P}}_{\overline{{\operatorname{ran}}\,}(\operatorname{Id}-\widetilde{T})}(0)=-v$. Let $x\in X$. Then the following hold:*

1. (i)

$(\forall{n\in{\mathbb{N}}})$* ${\operatorname{P}}_{U}\widetilde{T}^{n}={\operatorname{P}}_{U}T^{n}{\operatorname{R}}_{U}$.* 2. (ii)

$\widetilde{T}^{n}x-\widetilde{T}^{n+1}x={\operatorname{P}}_{g}\widetilde{T}^{n}x-2{\operatorname{P}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}x+{\operatorname{P}}_{U}\widetilde{T}^{n}x={\operatorname{P}}_{U}\widetilde{T}^{n}x-{\operatorname{R}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}x\to-v$. 3. (iii)

${\operatorname{P}}_{U}\widetilde{T}^{n}x-{\operatorname{P}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}x\to{\operatorname{P}}_{U}(-v)=0$. 4. (iv)

${\operatorname{P}}_{U}T^{n}x\>{\rightharpoonup}\>{\operatorname{P}}_{U}y(x)\in\operatorname{argmin}(\iota_{U}+g(\cdot-v))$. 5. (v)

${\operatorname{P}}_{U}\widetilde{T}^{n}x\>{\rightharpoonup}\>{\operatorname{P}}_{U}y({\operatorname{R}}_{U}x)\in\operatorname{argmin}(\iota_{U}+g(\cdot-v))$. 6. (vi)

${\operatorname{P}}_{g}\widetilde{T}^{n}x\>{\rightharpoonup}\>{\operatorname{P}}_{U}y({\operatorname{R}}_{U}x)-v\in\operatorname{dom}g$.

Proof. Observe that ${\operatorname{P}}_{U}{\operatorname{R}}_{U}={\operatorname{P}}_{U}$ and ${\operatorname{R}}_{U}^{2}=\operatorname{Id}$. (i): Using [14, Theorem 2.7(i)] we learn that $(\forall{n\in{\mathbb{N}}})$ ${\operatorname{P}}_{U}\widetilde{T}^{n}={\operatorname{P}}_{U}{\operatorname{R}}_{U}\widetilde{T}^{n}{\operatorname{R}}_{U}{\operatorname{R}}_{U}={\operatorname{P}}_{U}T^{n}{\operatorname{R}}_{U}$. (ii): $\widetilde{T}^{n}-\widetilde{T}^{n+1}={\operatorname{P}}_{g}\widetilde{T}^{n}-{\operatorname{P}}_{U}{\operatorname{R}}_{g}\widetilde{T}^{n}={\operatorname{P}}_{g}\widetilde{T}^{n}-2{\operatorname{P}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}+{\operatorname{P}}_{U}\widetilde{T}^{n}={\operatorname{P}}_{U}\widetilde{T}^{n}-{\operatorname{R}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}$. Now combine with 17. (iii): Recall that $-v\in U^{\perp}$ by 29. Now combine with (ii). (iv): This is Theorem 5.1. (v): Combine (i) and (iv) with $x$ replaced by ${\operatorname{R}}_{U}x$. (vi): It follows from (iii) and (v) that ${\operatorname{P}}_{U}{\operatorname{P}}_{g}\widetilde{T}^{n}x\>{\rightharpoonup}\>{\operatorname{P}}_{U}y({\operatorname{R}}_{U}x)$. Now combine with (ii). $\hfill\quad\blacksquare$

In the setting of Theorem 5.1, we point out that no general conclusion can be drawn about the sequence $({\operatorname{P}}_{g}T^{n}x)_{n\in{\mathbb{N}}}$ as we illustrate below.

Example 5.9** ($({\operatorname{P}}_{g}T^{n}x)_{n\in{\mathbb{N}}}$ may converge).**

Suppose that $(U,g)=(X,\iota_{X})$. Then ${\operatorname{P}}_{U}={\operatorname{P}}_{g}=T=\widetilde{T}=\operatorname{Id}$. Hence, ${\operatorname{ran}}\,(\operatorname{Id}-T)={\operatorname{ran}}\,(\operatorname{Id}-\widetilde{T})=\{0\}$. Consequently, $v=-v=0$ and $(\forall{n\in{\mathbb{N}}})$ $(\forall x\in X)$ ${\operatorname{P}}_{g}T^{n}x=x=\lim_{n\to\infty}{\operatorname{P}}_{g}T^{n}x$.

Example 5.10** ($({\operatorname{P}}_{g}T^{n}x)_{n\in{\mathbb{N}}}$ may have no cluster points).**

Suppose that $X=\mathbb{R}^{2}$, that $U=\mathbb{R}\times\{0\}$, that $C=\operatorname{epi}(\lvert\cdot\rvert+1)$ and that $g=\iota_{C}$. Let $x\in\left[-1,1\right]\times\{0\}$. Using induction, one can show that $(\forall n\in\{1,2,\ldots\})$ $T^{n}x=(0,n)\in C$. Consequently, $\lVert{\operatorname{P}}_{g}T^{n}x\rVert=\lVert{\operatorname{P}}_{C}T^{n}x\rVert=n\to+\infty$.

6 Minimizing the sum of finitely many functions

In this section we assume for simplicity that

[TABLE]

that $m\in\{2,3,\ldots\}$, that $I=\{1,2,\ldots,m\}$, and that

[TABLE]

for every $i\in I$. Furthermore, we set (see also [6] and [16])

[TABLE]

Remark 6.1**.**

In passing we point out that, by [11, Theorem 2.16], we have $(\forall i\in I)$ $D_{i}=\overline{\operatorname{dom}}\ \partial g_{i}=\overline{\operatorname{dom}}\ g_{i}$.

Fact 6.2**.**

Write $\mathbf{x}=(x_{i})_{i\in I}\in\mathbf{X}$. Then the following hold:

1. (i)

${\bf g}\colon\mathbf{X}\to\left]-\infty,+\infty\right]$* is convex, lower semicontinuous, and proper.* 2. (ii)

${\bf g}^{*}=\bigoplus_{i\in I}g_{i}^{*}$. 3. (iii)

$\partial{\bf g}=\bigtimes_{i\in I}\partial g_{i}$. 4. (iv)

{\operatorname{P}}_{{\bf\Delta}}\mathbf{x}=\mathbf{j}\big{(}\tfrac{1}{m}\sum_{i\in I}x_{i}\big{)}. 5. (v)

${\operatorname{P}}_{{\bf g}}=\bigtimes_{i\in I}{\operatorname{P}}_{g_{i}}$. 6. (vi)

{\bf\Delta}^{\perp}=\big{\{}{\mathbf{u}\in\mathbf{X}}~{}\big{|}~{}{\sum_{i\in I}u_{i}=0}\big{\}}.

Proof. (i): Clear. (ii): This is [6, Proposition 13.30]. (iii): This is [6, Proposition 16.9]. (iv): This is [6, Proposition 26.4(ii)]. (v): This is [6, Proposition 24.11]. (vi): This is [6, Proposition 26.4(i)]. $\hfill\quad\blacksquare$

Next we define the set of least squares solutions of $({D}_{i})_{i\in I}$

[TABLE]

Finally, throughout the remainder of this section, we assume that

[TABLE]

Remark 6.3**.**

In many applications, the individual functions $g_{i}$ have minimizers. In such cases, $(\forall i\in I)$ $0\in\operatorname{dom}\partial g_{i}^{*}\subseteq\operatorname{dom}g_{i}^{*}$, and therefore ${\boldsymbol{0}}\in\operatorname{dom}{\bf g}^{*}\subseteq{\bf\Delta}^{\perp}+\operatorname{dom}{\bf g}^{*}$.

Proposition 6.4**.**

The following hold:

1. (i)

${\bf v}={\operatorname{P}}_{\overline{{\bf\Delta}-\operatorname{dom}{\bf g}}}({\boldsymbol{0}})={\operatorname{P}}_{\overline{{\bf\Delta}-{{\bf D}}}}({\boldsymbol{0}})\in{\bf\Delta}^{\perp}$. 2. (ii)

$\operatorname{Fix}{\operatorname{P}}_{{\bf\Delta}}{\operatorname{P}}_{{{\bf D}}}={\bf\Delta}\cap({\bf v}+{{\bf D}})\neq\varnothing$. 3. (iii)

$(\forall y\in\operatorname{Fix}{\operatorname{P}}_{{\bf\Delta}}{\operatorname{P}}_{{{\bf D}}})$* ${\bf v}=\mathbf{y}-{\operatorname{P}}_{{{\bf D}}}(\mathbf{y})$.* 4. (iv)

\mathbf{Z}=\big{\{}{\mathbf{x}\in{\bf\Delta}}~{}\big{|}~{}{{\bf\Delta}^{\perp}\cap\partial{\bf g}(\mathbf{x}-{\bf v})\neq\varnothing}\big{\}}=\mathbf{j}\big{(}\operatorname{zer}\sum_{i\in I}\partial g_{i}(\cdot-v_{i})\big{)}. 5. (v)

\operatorname{zer}\Big{(}\sum_{i\in I}\partial g_{i}(\cdot-v_{i})\Big{)}\neq\varnothing. 6. (vi)

{L}=\operatorname{Fix}\Big{(}\tfrac{1}{m}\sum_{i\in I}{\operatorname{P}}_{{D}_{i}}\Big{)}=\bigcap_{i\in I}(v_{i}+{D}_{i}).** 7. (vii)

e(\mathbf{Z})=\operatorname{zer}\big{(}\sum_{i\in I}\partial g_{i}(\cdot-v_{i})\big{)}\subseteq\cap_{i\in I}(\operatorname{dom}\partial g_{i}(\cdot-v_{i}))\subseteq\cap_{i\in I}(v_{i}+{D}_{i})={L}.

Proof. (i): Observe that that $\overline{{\bf\Delta}-\operatorname{dom}{\bf g}}=\overline{{\bf\Delta}-\overline{\operatorname{dom}}{\bf g}}=\overline{{\bf\Delta}-{\bf D}}$. Now combine this with 74 and Proposition 3.1(ii) applied with $(X,U,g)$ replaced by $(\mathbf{X},{\bf\Delta},{\bf g})$. (ii)&(iii): Combine [3, Lemma 2.2(i)&(iv)] and 34 applied with $(X,U,g)$ replaced by $(\mathbf{X},{\bf\Delta},{\bf g})$. (iv): The first identity follows from applying 30 with $(X,U,g)$ replaced by $(\mathbf{X},{\bf\Delta},{\bf g})$. The second identity follows from [6, Proposition 26.4(vii)&(viii)]. (v): This is a direct consequence of item (iv). (vi): Combine item (i), [3, Lemma 2.2(i)] and [8, Corollary 3.1]. (vii): This is a direct consequence of (iv) and (vi). $\hfill\quad\blacksquare$

Proposition 6.5**.**

Suppose that $j\in I$ satisfies that $\operatorname{dom}g_{j}=X$. Then $v_{j}=0$.

Proof. Set ${\bf A}=\operatorname{argmin}(\iota_{{\bf\Delta}}+{\bf g}(\cdot-{\bf v}))$ and observe that Proposition 6.4(i)&(ii) imply that ${\bf A}\subseteq{\bf\Delta}\cap({\bf v}+\operatorname{dom}{\bf g})\subseteq{\bf\Delta}\cap({\bf v}+{{\bf D}})=\operatorname{Fix}{\operatorname{P}}_{{\bf\Delta}}{\operatorname{P}}_{{{\bf D}}}$. Note that 74 and Theorem 3.4 (applied with $(U,g)$ replaced by $({\bf\Delta},{\bf g})$) imply that ${\bf A}=\mathbf{Z}$. Hence, $e({\bf A})=e(\mathbf{Z})\subseteq{L}$, by Proposition 6.4(vii). Now, let $\mathbf{y}\in\operatorname{Fix}{\operatorname{P}}_{{\bf\Delta}}{\operatorname{P}}_{{{\bf D}}}$. Then Proposition 6.4(iii) implies that ${\bf v}=\mathbf{y}-{\operatorname{P}}_{{{\bf D}}}(\mathbf{y})=(y_{1},\ldots,y_{m})-({\operatorname{P}}_{{D}_{1}}y_{1},\ldots,{\operatorname{P}}_{{D}_{m}}y_{m})$. Consequently, if $D_{j}=X$ then $v_{j}=y_{j}-{\operatorname{P}}_{{D}_{j}}y_{j}=0$. $\hfill\quad\blacksquare$

Theorem 6.6**.**

Let $\mathbf{x}=(x_{i})_{i\in I}\in\mathbf{X}$ and set $\mathbf{y}=\lim_{n\to\infty}{\operatorname{P}}_{\operatorname{Fix}{\bf T}}(n{\bf v}+{\bf T}^{n}\mathbf{x})$. Then

[TABLE]

[TABLE]

Furthermore,

[TABLE]

Proof. 75 and 76 follow from applying Theorem 5.1 with $(X,U,g)$ replaced by $(\mathbf{X},{\bf\Delta},{\bf g})$. It follows from combining 75 and Theorem 3.4 (applied with $(U,g)$ replaced by $({\bf\Delta},{\bf g})$) that ${\operatorname{P}}_{{\bf\Delta}}\mathbf{y}\in\operatorname{argmin}(\iota_{\bf\Delta}+{\bf g}(\cdot-{\bf v}))=\mathbf{Z}$. Now combine with Proposition 6.4(vii). $\hfill\quad\blacksquare$

Corollary 6.7**.**

Let $x_{0}\in X$, and set $\overline{x}_{0}=x_{0,1}=\cdots=x_{0,m}=x_{0}$. Update via $(\forall{n\in{\mathbb{N}}})$

[TABLE]

Then \overline{x}_{n}\to\overline{x}\in\operatorname{argmin}\big{(}\sum_{i\in I}g_{i}(\cdot-v_{i})\big{)}.

Proof. Combine Theorem 6.6 and Proposition 6.4(v)&(iv)&(v) in view of 74. $\hfill\quad\blacksquare$

Corollary 6.8**.**

Suppose that $J\subseteq I$, that for every $i\in I\smallsetminus J$, $f_{i}\colon X\to\mathbb{R}$ is convex and satisfies $\operatorname{dom}f_{i}=X$ and $\operatorname{argmin}f_{i}\neq\varnothing$, and that for every $i\in J$, $C_{i}\neq X$ is convex, closed, and nonempty. Set ${L}_{C}=\operatorname{argmin}\sum_{i\in J}{\operatorname{dist}}_{C_{i}}^{2}$. Consider the problem

[TABLE]

Suppose that \operatorname{zer}\big{(}\sum_{i\in I\smallsetminus J}\partial f_{i}+\sum_{i\in J}{\operatorname{N}}_{C_{i}}(\cdot-v_{i})\big{)}\neq\varnothing. Let $x_{0}\in X$, and set $\overline{x}_{0}=x_{0,1}=\cdots=x_{0,m}=x_{0}$. Update via $(\forall{n\in{\mathbb{N}}})$

[TABLE]

Then $\overline{x}_{n}\to\overline{x}\in X$, and $\overline{x}$ is a solution of

[TABLE]

In particular, if $\cap_{i\in J}C_{i}\neq\varnothing$, then ${L}_{C}=\cap_{i\in J}C_{i}\neq\varnothing$ and $\overline{x}$ is a solution of 79.

Proof. Suppose that $g_{i}=f_{i}$, if $i\in I\smallsetminus J$; and $g_{i}=\iota_{C_{i}}$, if $i\in J$, and observe that 79 reduces to

[TABLE]

Note that combining 78 and [6, Example 23.4] yields 80. It follows from Proposition 6.5 that $(\forall i\in I\smallsetminus J)$ $v_{i}=0$. Consequently, \operatorname{zer}\big{(}\sum_{i\in I}\partial g_{i}(\cdot-v_{i})\big{)}=\operatorname{zer}\big{(}\sum_{i\in I\smallsetminus J}\partial f_{i}+\sum_{i\in J}{\operatorname{N}}_{C_{i}}(\cdot-v_{i})\big{)}\neq\varnothing, and by Corollary 6.7 we have $\overline{x}_{n}\to\overline{x}\in X$, and \overline{x}\in\operatorname{zer}\big{(}\sum_{i\in I\smallsetminus J}\partial f_{i}+\sum_{i\in J}{\operatorname{N}}_{C_{i}}(\cdot-v_{i})\big{)}. Finally, using Proposition 6.4(vi), $(\exists u\in X)$ $-u\in\sum_{i\in I\smallsetminus J}\partial f_{i}(\overline{x})=\partial(\sum_{i\in I\smallsetminus J}f_{i})(\overline{x})$ and $u\in\sum_{i\in J}{\operatorname{N}}_{C_{i}}(\overline{x}-v_{i})\subseteq{\operatorname{N}}_{\cap_{i\in J}(v_{i}+C_{i})}(\overline{x})={\operatorname{N}}_{{L}_{C}}(\overline{x})$. Therefore, $\overline{x}$ solves 81. $\hfill\quad\blacksquare$

Acknowledgements

The authors thank the editor and three anonymous referees for insightful comments that led to a substantially improved manuscript. The research of HHB was partially supported by a Discovery Grant of the Natural Sciences and Engineering Research Council of Canada. The research of WMM was partially supported by the Natural Sciences and Engineering Research Council of Canada Postdoctoral Fellowship.