Uniqueness of DRS as the 2 Operator Resolvent-Splitting and Impossibility of 3 Operator Resolvent-Splitting

TL;DR

This paper proves the uniqueness of Douglas--Rachford splitting (DRS) as the only 2-operator resolvent-splitting with certain properties and shows that direct generalization to 3 operators is impossible without lifting, providing a minimal lifting solution.

Contribution

It establishes the uniqueness of DRS among 2-operator resolvent-splittings and introduces a minimal lifting method for 3-operator generalization.

Findings

DRS is the unique 2-operator resolvent-splitting with favorable properties.

Direct 3-operator generalization of DRS is impossible without lifting.

A novel minimal lifting 3-operator resolvent-splitting is proposed.

Abstract

Given the success of Douglas--Rachford splitting (DRS), it is natural to ask whether DRS can be generalized. Are there other 2 operator resolvent-splittings sharing the favorable properties of DRS? Can DRS be generalized to 3 operators? This work presents the answers: no and no. In a certain sense, DRS is the unique 2 operator resolvent-splitting, and generalizing DRS to 3 operators is impossible without lifting, where lifting roughly corresponds to enlarging the problem size. The impossibility result further raises a question. How much lifting is necessary to generalize DRS to 3 operators? This work presents the answer by providing a novel 3 operator resolvent-splitting with provably minimal lifting that directly generalizes DRS.

Equations

$$\underset{x\in {\mathbb{R}}^d}{\text{mbox}find}0\in (A+B)x$$

$$z^{k+1}=(1-\theta /2)z^k+(\theta /2)(2J_{\alpha A}-I)(2J_{\alpha B}-I)z^k$$

$$x^{k+1}=J_{\alpha A}{x}^k$$

$$x^{k+1}=J_{\alpha B}(I-\alpha A)x^k,$$

$$x^{k+1}$$

$$u^{k+1}$$

$$z^{k+1}=(I-J_{\alpha B}+J_{\alpha A}\circ (2J_{\alpha B}-I-\alpha C\circ J_{\alpha B}))z^k$$

$$J_{\alpha A}=(I+\alpha A{)}^{-1}$$

$$\Vert Tx-Ty{\Vert }^2\le \Vert x-y{\Vert }^2$$

$$\Vert Fx-Fy{\Vert }^2\le ⟨x-y,Fx-Fy⟩$$

$$z^{k+1}=T{z}^k$$

$$z^{k+1}=(2J_{\alpha A}-I)(2J_{\alpha B}-I)z^k$$

$$z^{k+1}=(1-\theta /2)z^k+(\theta /2)(2J_{\alpha A}-I)(2J_{\alpha B}-I)z^k.$$

$$\underset{x\in {\mathbb{R}}^d}{\text{mbox}find}0\in (A+B)x$$

$$\exists z^{\star}\in\mathbb{R}^{d^{\prime}}\text{ such that}\,\left(\begin{array}[]{ll}T(A,B,z^{\star})&=z^{\star}\\ S(A,B,z^{\star})&=x^{\star}\end{array}\right)\quad\Leftrightarrow\quad 0\in(A+B)(x^{\star})$$

$$T:\mathcal{M}({\mathbb{R}}^d)\times \mathcal{M}({\mathbb{R}}^d)\times {\mathbb{R}}^{d^{\prime }}\to {\mathbb{R}}^{d^{\prime }}$$

$$S:\mathcal{M}({\mathbb{R}}^d)\times \mathcal{M}({\mathbb{R}}^d)\times {\mathbb{R}}^{d^{\prime }}\to {\mathbb{R}}^d,$$

$$T(A,B,z)$$

$$S(A,B,z)$$

$${\mathcal{F}}_0=\{I\}\cup \{J_{\alpha ,1}|\alpha >0\}\cup \{J_{\beta ,2}|\beta >0\}.$$

$${\mathcal{F}}_{i+1}=\{F+G|F,G\in {\mathcal{F}}_i\}\cup \{F\circ G|F,G\in {\mathcal{F}}_i\}\cup \{\gamma F|F\in {\mathcal{F}}_i,\gamma \in \mathbb{R}\}$$

$$(F\circ G)(A,B,z)=F(A,B,G(A,B,z))$$

$$\mathcal{F}=\bigcup _{i=0}^{\infty }{\mathcal{F}}_i.$$

$$T^k{z}^0\to z^{\star },S{z}^{\star }\in \mathrm{zer}(A+B)$$

$$T^k=\underset{\text{k times}}{\underbrace{T\circ T\circ \cdots \circ T}}.$$

$$z^{k+1}=T(z^k)\iff a{z}^{k+1}=aT(a^{-1}a{z}^k)$$

$$(T(A,B,\cdot ),S(A,B,\cdot ))\iff (T(B,A,\cdot ),S(B,A,\cdot )).$$

$$x_1$$

$$x_2$$

$$T(z)$$

$$S(z)$$

$$\tilde{T}(A,B,z)=T(B,A,z),\tilde{S}(A,B,Z)=S(B,A,z),$$

$$0=\left[\begin{array}{ccccccccc}\ast & 1& 0& 0& 0& 0& 0& 0& 0\\ 0& -1& 1& 0& 0& 0& \alpha & 0& 0\\ \ast & \ast & \ast & 1& 0& 0& 0& 0& 0\\ 0& 0& 0& -1& 1& 0& 0& \beta & 0\\ \ast & \ast & \ast & \ast & \ast & 1& \ast & \ast & 0\\ \ast & \ast & \ast & \ast & \ast & \ast & \ast & \ast & 1\end{array}\right]\left[\begin{array}{c}{z}_0\\ z_1\\ x_1\\ z_2\\ x_2\\ T{z}_0\\ \tilde{A}{x}_1\\ \tilde{B}{x}_2\\ S{z}_0\end{array}\right].$$

Full text

1 Introduction

In 1979, Lions and Mercier presented Douglas–Rachford splitting (DRS) which solves the monotone inclusion problem

[TABLE]

with

[TABLE]

for any $\alpha>0$ and $\theta\in(0,2)$, where $A$ and $B$ are maximal monotone operators and $J_{\alpha A}$ and $J_{\alpha B}$ are their resolvents peaceman1955 ; douglas1956 ; lions1979 . Since its introduction, DRS has enjoyed great popularity and has provided great value to the field of optimization.

Given the success of DRS, one may ask the following two questions:

Are there other 2 operator resolvent-splittings? 2. 2.

Can we generalize DRS to 3 operators?

In fact, the second question has been a long-standing open problem posed by Lions and Mercier themselves: “[T]he convergence seems difficult to prove … in the case of a sum of 3 operators.” After all, identifying why a tool works and generalizing it is a common and often fruitful exercise in mathematics.

This work presents the answers to these questions: no and no. In a certain sense, DRS is the unique 2 operator resolvent-splitting. In a certain sense, there is no 3 operator resolvent-splitting without lifting, where lifting roughly corresponds to enlarging the problem size.

This impossibility result further raises the following question:

To generalize DRS to 3 operators, how much lifting is necessary?

This work presents the answer by providing a novel 3 operator resolvent-splitting with provably minimal lifting.

Background.

To discuss what constitutes a generalization of DRS, we first point out a few key properties of DRS. Perhaps a generalization of DRS should satisfy these as well.

DRS is a resolvent-splitting in that it is constructed with scalar multiplication, addition, and resolvents. 2. 2.

DRS is frugal in that it uses $J_{\alpha A}$ and $J_{\alpha B}$ only once per iteration. 3. 3.

DRS converges unconditionally in that it works for any maximal monotone $A$ and $B$. 4. 4.

DRS uses no lifting in that the fixed-point mapping maps from $\mathbb{R}^{d}$ to $\mathbb{R}^{d}$, where $x\in\mathbb{R}^{d}$. In other words, DRS does not enlarge the problem size.

Consider the proximal point method (PPM) martinet1970 ; martinet1972 ; rockafellar1976 ; brezis1978 , which finds an $x\in\mathbb{R}^{d}$ such that $0\in Ax$ with

[TABLE]

for any $\alpha>0$ and maximal monotone $A$. DRS generalizes PPM, and both methods are frugal, converge unconditionally, use no lifting, and rely on resolvents. Therefore, to require the 4 properties in a generalization of DRS seems reasonable.

Many other splittings have been presented since DRS, and they have certainly provided great value to the field of optimization. These splittings solve a wide range of different problem classes and are designed to be effective under a wide range of different computational considerations. Many of them include DRS as a special case and therefore are generalizations of DRS, in that sense. However, they do not satisfy the 4 stated properties and therefore are not generalizations of DRS, in this sense.

Forward-backward splitting (FBS) passty1979 ,

[TABLE]

which requires $A$ to be cocoercive, is frugal, uses no lifting, but is not a resolvent-splitting. Primal-dual hybrid gradient method (PDHG) zhu08 ; pock2009 ; esser2010 ; ChaPoc11 , also known as Chambolle–Pock,

[TABLE]

is frugal but uses lifting. Davis–Yin splitting (DYS) davis2017 , which finds an $x\in\mathbb{R}^{d}$ such that $0\in(A+B+C)x$, where $C$ is cocoercive,

[TABLE]

is frugal, uses no lifting, but is not a resolvent-splitting. Other methods, such as FBFS tseng2000 , PPXA combettes2008 ; combettes2011 , PDFP2O/PAPC loris2011 ; chen2013 ; driori2015 , RFBS banert2012 , Condat–Vũ condat2013 ; vu2013 , GFBS raguet2013 , PD3O yan2016 , PDFP Chen2016 , AFBA latafat2017 , FBHFS ba2017 , FDRS BAFDRS ; Raguet2018 , FRB malitsky2018 , projective splitting Eckstein2017 ; Combettes2018 ; johnstone2018 ; johnstone2019 , and the methods of ba2011 ; Combettes2012 ; bot_primal-dual_2013 ; combettes2014 all fail to satisfy the 4 properties.

Organization of the paper.

In Section 2, we show that DRS is the only frugal, unconditionally convergent resolvent-splitting without lifting for the 2 operator problem. We do so by characterizing all frugal resolvent-splittings without lifting and showing that DRS is the only one among them that unconditionally converges.

In Section 3, we show that there is no resolvent-splitting without lifting for the 3 operator problem, even if the splitting is not frugal and not convergent. In particular, we show such a scheme without lifting cannot be a fixed-point encoding.

In Section 4, we define and quantify the notion of lifting for the 3 operator problem. We then provide a novel frugal, unconditionally convergent resolvent-splitting with provably minimal lifting for the 3 operator problem that directly generalizes DRS.

Definitions.

We briefly review some standard notation and results of operator theory. Interested readers can find in-depth discussion of these concepts in standard references such as ryu2016 ; BauschkeCombettes2017_convex .

Write $\langle\cdot,\cdot\rangle$ for the standard Euclidean inner product in $\mathbb{R}^{d}$. We say $A$ is an operator on $\mathbb{R}^{d}$ if $A$ maps points of $\mathbb{R}^{d}$ to subsets of $\mathbb{R}^{d}$. Given a matrix $M\in\mathbb{R}^{d\times d}$ also write $M:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ to denote the linear operator defined by the matrix $M$. In particular, write $I$ for both the identity operator and the identity matrix. Write $\mathcal{M}(\mathbb{R}^{d})$ for the set of all maximal monotone operators on $\mathbb{R}^{d}$. For any maximal monotone operator $A$ and $\alpha>0$, write

[TABLE]

for the resolvent of $A$. A mapping $T:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is nonexpansive if

[TABLE]

for all $x,y\in\mathbb{R}^{d}$. A mapping $F:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is firmly nonexpansive if

[TABLE]

for all $x,y\in\mathbb{R}^{d}$. Resolvents are firmly nonexpansive. Given a mapping $T:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and a starting point $z^{0}\in\mathbb{R}^{d}$, we call

[TABLE]

the fixed-point iteration with respect to $T$. A fixed-point iteration with respect to a nonexpansive mapping need not converge. A mapping $T:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is averaged if it can be expressed as $T=(1-\theta)I+\theta R$, where $R:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is nonexpansive and $\theta\in(0,1)$. Note that $R$ and $T$ share the same fixed points. The fixed-point iteration with respect to an averaged mapping $T:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ converges in that $z^{k}\rightarrow z^{\star}$ where $Tz^{\star}=z^{\star}$, if a fixed point exists.

For any $A\in\mathcal{M}(\mathbb{R}^{d})$, write $\mathop{\bf zer}A=\{x\,|\,0\in Ax\}$ for the set of zeros of $A$. Consider the monotone inclusion problem of finding an element of $\mathop{\bf zer}(A+B)$, where $A,B\in\mathcal{M}(\mathbb{R}^{d})$. Peaceman–Rachford splitting (PRS) peaceman1955 ; lions1979 is the fixed-point iteration

[TABLE]

with $\alpha>0$. PRS is not guaranteed to converge. Douglas–Rachford splitting (DRS) is the fixed-point iteration

[TABLE]

with $\alpha>0$ and $\theta\in(0,2)$. (Some may call this “relaxed PRS”.) DRS is guaranteed to converge in the sense that $z^{k}\rightarrow z^{\star}$ for some $z^{\star}$ where $J_{\alpha B}z^{\star}\in\mathop{\bf zer}(A+B)$, if $\mathop{\bf zer}(A+B)$ is not empty.

2 Uniqueness of DRS as the unique

frugal, unconditionally convergent 2 operator resolvent-splitting without lifting

In this section, we define what a frugal, unconditionally convergent 2 operator resolvent-splitting without lifting is and prove DRS is the only such splitting.

2.1 Definitions

When reading the definitions, it is helpful to think of DRS as a specific example. In the terminology and notation we soon establish, DRS is an unconditionally convergent frugal resolvent-splitting without lifting and $d^{\prime}=d$, $T(A,B,z)=(1-\theta/2)I+(\theta/2)(2J_{\alpha A}-I)(2J_{\alpha B}-I)$, and $S(A,B,z)=J_{\alpha B}$.

Given a dimension $d$, define the problem class (2op-$\mathbb{R}^{d}$) to be the collection of monotone inclusion problems of the form

[TABLE]

with $A,B\in\mathcal{M}(\mathbb{R}^{d})$.

Fixed-point encoding.

A pair of functions $(T,S)$ is a fixed-point encoding for the problem class (2op-$\mathbb{R}^{d}$) if

[TABLE]

for all $A,B\in\mathcal{M}(\mathbb{R}^{d})$. We call

[TABLE]

the fixed-point mapping and

[TABLE]

the solution mapping. To clarify, a fixed-point encoding is defined for the entire problem class (2op-$\mathbb{R}^{d}$), rather than a single instance of the monotone inclusion problem.

When we fix $A,B\in\mathcal{M}(\mathbb{R}^{d})$, fixed points of $T(A,B,\cdot):\mathbb{R}^{d^{\prime}}\rightarrow\mathbb{R}^{d^{\prime}}$ corresponds to zeros of $A+B$. We say that points in $\mathop{\bf zer}(A+B)$ are encoded as fixed points of $T(A,B,\cdot)$. For notational simplicity, we often drop the dependency on $A$ and $B$ and write $Tz$ and $Sz$ for $T(A,B,z)$ and $S(A,B,z)$.

In this section, we only consider $d^{\prime}=d$, as we limit our attention to fixed-point encodings without lifting (formally defined soon). In general, however, the dimension $d$ of problems in (2op-$\mathbb{R}^{d}$) and the dimension $d^{\prime}$ of the fixed-point mapping need not be the same. The purpose of allowing $d^{\prime}\neq d$ will become clearer later in Section 4, where an analogously defined $d^{\prime}$ is larger than $d$.

Under this definition, DRS is a collection of fixed-point encodings. For each choice of $d$, $\alpha>0$, $\theta\in(0,2)$, and $\eta\in\mathbb{R}$ the pair of functions $(T,S)$ defined by

[TABLE]

is a instance of DRS and it is a fixed-point encoding for the problem class (2op-$\mathbb{R}^{d}$).

Frugal resolvent-splitting without lifting.

Loosely speaking, $(T,S)$ is a resolvent-splitting for the problem class (2op-$\mathbb{R}^{d}$) if it is a fixed-point encoding constructed with resolvents of $A$ and $B$, addition, and scalar multiplication. Loosely speaking, $(T,S)$ is frugal if it uses $J_{\alpha A}$ and $J_{\beta B}$ once, in that a single evaluation of $J_{\alpha A}$ and a single evaluation of $J_{\beta B}$ is used to evaluate both $Tz$ and $Sz$ for any $z$. $(T,S)$ is without lifting if $T(A,B,\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and $S(A,B,\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ for any $A,B\in\mathcal{M}(\mathbb{R}^{d})$, i.e., if $d^{\prime}=d$.

We now make the definitions precise. Let $I$ be the “identity mapping” defined as $I:\mathcal{M}(\mathbb{R}^{d})\times\mathcal{M}(\mathbb{R}^{d})\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and $I(A,B,z)=z$ for any $A,B\in\mathcal{M}(\mathbb{R}^{d})$ and $z\in\mathbb{R}^{d}$. Let $J_{\alpha,1}$ be the resolvent with respect to the first operator defined as $J_{\alpha,1}:\mathcal{M}(\mathbb{R}^{d})\times\mathcal{M}(\mathbb{R}^{d})\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and $J_{\alpha,1}(A,B,z)=J_{\alpha A}(z)$ for any $A,B\in\mathcal{M}(\mathbb{R}^{d})$ and $z\in\mathbb{R}^{d}$. Define $J_{\beta,2}$ likewise with $J_{\beta,2}(A,B,z)=J_{\beta B}(z)$. Define the class of mappings

[TABLE]

Recursively define

[TABLE]

for $i=0,1,2,\dots$. The “composition” $F\circ G$ is defined with

[TABLE]

for any $z\in\mathbb{R}^{d}$ and $A,B\in\mathcal{M}(\mathbb{R}^{d})$. Note that $\mathcal{F}_{0}\subset\mathcal{F}_{1}\subset\mathcal{F}_{2}\subset\cdots$. Finally define

[TABLE]

To clarify, elements of $\mathcal{F}$ map $\mathcal{M}(\mathbb{R}^{d})\times\mathcal{M}(\mathbb{R}^{d})\times\mathbb{R}^{d}$ to $\mathbb{R}^{d}$. If $R\in\mathcal{F}$ and $A,B\in\mathcal{M}(\mathbb{R}^{d})$, then $R(A,B,\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$. These mappings are constructed with (finitely many) resolvents of $A$ and $B$, addition, and scalar multiplication.

As an aside, we could have defined $\mathcal{F}$ as the “near-ring” generated by $J_{\alpha,1}$ and $J_{\beta,2}$ for all $\alpha>0$ and $\beta>0$ and $\gamma I$ for all $\gamma\in\mathbb{R}$. The set is not a ring because $T\circ(U+V)\neq T\circ U+T\circ V$ for non-linear functions.

We say $(T,S)$ is a resolvent-splitting without lifting for the problem class (2op-$\mathbb{R}^{d}$), if $(T,S)$ is a fixed-point encoding for the problem class (2op-$\mathbb{R}^{d}$), and $T,S\in\mathcal{F}$. (Remember, $T\in\mathcal{F}$ implies $T(A,B,\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$.)

When $T,S\in\mathcal{F}$, one can evaluate $T(A,B,z)$ and $S(A,B,z)$ for given $z\in\mathbb{R}^{d}$ and $A,B\in\mathcal{M}(\mathbb{R}^{d})$ in finitely many steps, where each step is scalar multiplication, vector addition, or a resolvent evaluation. We say $(T,S)$ is frugal if it has a step-by-step (serial) evaluation procedure such that exactly one step computes $J_{\alpha A}$, exactly one step computes $J_{\beta B}$, and both $T(A,B,z)$ and $S(A,B,z)$ are output at the end.

Unconditional convergence.

We say $(T,S)$ converges unconditionally for the problem class (2op-$\mathbb{R}^{d}$) if

[TABLE]

for any $z^{0}\in\mathbb{R}^{d^{\prime}}$ and $A,B\in\mathcal{M}(\mathbb{R}^{d})$ as $k\rightarrow\infty$, when $\mathop{\bf zer}(A+B)\neq\emptyset$. To clarify,

[TABLE]

We say the convergence is unconditional because there are no conditions on the operators $A,B\in\mathcal{M}(\mathbb{R}^{d})$ or the starting point $z^{0}\in\mathbb{R}^{d}$.

For example, with DRS, the $z^{k}$-iterates do not, in general, converge to a solution. Rather, $z^{k}\rightarrow z^{\star}$, where $J_{\alpha B}z^{\star}$ is a solution to the monotone inclusion problem, when a solution exists.

The notion of unconditional convergence is unrelated to weak and strong convergence. In infinite dimensional spaces, we would require the convergence $T^{k}z^{0}\rightarrow z^{\star}$ to hold weakly, but weak and strong convergence coincide in finite dimensions. We avoid infinite dimensional spaces because defining the notion of lifting would be awkward when $d=\infty$.

Equivalence.

Given a fixed-point iteration, we can scale it with a nonzero scalar to get another one that is essentially the same, i.e.,

[TABLE]

for any $a\in\mathbb{R}$ such that $a\neq 0$. Given resolvent-splitting, we can swap the role of $A$ and $B$ to get another one that is conceptually no different, i.e.,

[TABLE]

Two resolvent-splittings without lifting are equivalent if one can be obtained from the other through scaling with a nonzero scalar and/or swapping the role of $A$ and $B$.

2.2 Uniqueness result

Theorem 2.1

Up to equivalence, $(T,S)$ is a frugal resolvent-splittings without lifting for the problem class (2op-$\mathbb{R}^{d}$) if and only if it is of the form

[TABLE]

for some $\alpha,\beta>0$, $\theta\neq 0$, and $\eta\in\mathbb{R}$.

Note that Theorem 2.1 says nothing about convergence. Theorem 2.2 characterizes the splittings of Theorem 2.1 that do converge.

Theorem 2.2

$(T,S)$* of Theorem 2.1 converges unconditionally if and only if $\alpha=\beta$ and $\theta\in(0,2)$ if $d\geq 2$.*

When $d=1$, the splittings $(T,S)$ of Theorem 2.1 may converge under more general conditions, but we do not pursue this discussion.

Corollary 1

Up to equivalence, the class of DRS splittings (the collection parameterized by $\alpha>0$, $\theta\in(0,2)$, and $\eta\in\mathbb{R}$) are the only frugal, unconditionally convergent resolvent-splittings without lifting for the problem class (2op-$\mathbb{R}^{d}$) when $d\geq 2$.

Proof

A frugal resolvent-splittings without lifting must be equivalent to a splitting of the form of Theorem 2.1. If it is furthermore unconditionally convergent, then, by Theorem 2.2, we have $\alpha=\beta$ and $\theta\in(0,2)$, which corresponds to DRS. ∎

2.3 Proof of Theorem 2.1

2.3.1 Outline

The main part of proof, which shows that any frugal resolvent-splitting without lifting is of the form of Theorem 2.1, can be divided into in roughly three steps. In the first step, we represent a given resolvent-splitting $(T,S)$ with a linear system of equations, and simplify the system using Gaussian elimination. In the second step, we show that the system of linear equalities must imply certain equalities one would expect from a fixed-point encoding. This is done by using a Farkas-type lemma to take a certain element from the null space of the linear system and using it to construct the counter example. In the third step, we use the conclusion of the second step to eliminate and characterize the parameters of $(T,S)$.

2.3.2 Proof

Showing that $(T,S)$ of Theorem 2.1 is indeed a fixed-point encoding is straightforward. Let $x^{\star}\in\mathbb{R}^{d}$ satisfy $0\in(A+B)x^{\star}$. Let $\tilde{A}x^{\star}\in Ax^{\star}$ and $\tilde{B}x^{\star}\in Bx^{\star}$ such that $\tilde{A}x^{\star}+\tilde{B}x^{\star}=0$, and let $z_{0}=x^{\star}+\alpha\tilde{A}x^{\star}$. Then $x_{1}=x_{2}=x^{\star}$, $Tz_{0}=z_{0}$, and $Sz_{0}=x^{\star}$. On the other hand, assume $T(A,B,z^{\star})=z^{\star}$. Then $x_{1}=x_{2}$. Write $x^{\star}=x_{1}=x_{2}$, $\tilde{A}x^{\star}=(1/\alpha)(z^{\star}-x^{\star})$, and $\tilde{B}x^{\star}=(1/\alpha)(x^{\star}-z^{\star})$. Then $\tilde{A}x^{\star}\in Ax^{\star}$, $\tilde{B}x^{\star}\in Bx^{\star}$, and $\tilde{A}x^{\star}+\tilde{B}x^{\star}=0$, which implies $x^{\star}=S(z^{\star})$ is a solution.

We now need to show the other direction, that any frugal resolvent-splitting without lifting for the problem class (2op-$\mathbb{R}^{d}$) is of the form of Theorem 2.1, up to equivalence.

First, we discuss the following Farkas-type lemma.

Lemma 1

Let $M\in\mathbb{R}^{m\times n}$ and $c\in\mathbb{R}^{n}$ be fixed coefficients, and let $v\in\mathbb{R}^{n}$ be a variable. If there is a $w\in\mathbb{R}^{m}$ such that $w^{T}M=c^{T}$ then the linear equalities $Mv=0$ imply the linear equality $c^{T}v=0$. If there is no such $w$, then there is an instance of $v\in\mathbb{R}^{n}$ such that $Mv=0$ but $c^{T}v\neq 0$.

An equivalent way to state Lemma 1 is to say that $Mv=0$ implies $c^{T}v=0$ if and only if we can linearly combine the rows of $Mv=0$ to obtain $c^{T}v=0$. We say Lemma 1 is of Farkas-type as it resembles Farkas’ result on systems of linear inequalities Farkas1902 . For a systematic study on Farkas-type theorems, see bot_farkas_2005 ; dinh_farkas_2007 . Lemma 1 can be directly and easily proved with standard linear algebra.

We now proceed onto the main proof. Let $(T,S)$ be a frugal resolvent-splitting without lifting.

Consider an evaluation procedure of $(T,S)$ that establishes frugality. In the step-by-step computation, either $J_{\alpha A}$ or $J_{\beta B}$ is evaluated before the other. Without loss of generality, assume $J_{\alpha A}$ is evaluated before $J_{\beta B}$ in this ordering, since we can otherwise consider $(\tilde{T},\tilde{S})$ defined with

[TABLE]

the equivalent splitting with the order of $A$ and $B$ swapped.

Consider the evaluation of $Tz_{0}$ and $Sz_{0}$ for $z_{0}\in\mathbb{R}^{d}$. Write $z_{1}$ and $z_{2}$ for the inputs and $x_{1}$ and $x_{2}$ for the outputs of the resolvent evaluations with respect to $A$ and $B$, i.e., $x_{1}=J_{\alpha A}z_{1}$ and $x_{2}=J_{\beta B}z_{2}$. Define $\tilde{A}x_{1}$ and $\tilde{B}x_{2}$ with $x_{1}+\alpha\tilde{A}x_{1}=z_{1}$ and $x_{2}+\beta\tilde{B}x_{2}=z_{2}$. By definition of resolvents, we have $\tilde{A}x_{1}\in Ax_{1}$ and $\tilde{B}x_{2}\in Bx_{2}$.

All computational steps except the evaluations of $J_{\alpha A}$ and $J_{\beta B}$ amount to forming linear combinations of previous information since scalar multiplication and vector addition are the only other operations allowed in $(T,S)$. Therefore, we can express the evaluation of $Tz_{0}$ and $Sz_{0}$ as

[TABLE]

Each scalar in the matrix represents a $d\times d$ block. The symbol $*$ denotes a fixed scalar coefficient that we have not yet parameterized. Row 1 defines $z_{1}$, the input to $J_{\alpha A}$. Row 2 represents $x_{1}=J_{\alpha A}z_{1}\,\Leftrightarrow\,x_{1}+\alpha Ax_{1}\ni z_{1}$. Row 3 defines $z_{2}$, the input to $J_{\beta B}$. Row 4 represents $x_{2}=J_{\beta B}z_{2}\,\Leftrightarrow\,x_{2}+\beta Bx_{2}\ni z_{2}$. Row 5 defines $Tz_{0}$. Row 6 defines $Sz_{0}$. We will simplify the system first and then explicitly parameterize the coefficients to keep the notation tractable.

Next, we simplify the system (1). Permute the rows of (1) to get the equivalent linear system

[TABLE]

Since permuting the rows is a reversible process, (1) and (2) are equivalent. In the step-by-step evaluation procedure of $(T,S)$, the evaluation of $T$ or $S$ completes before the other. As the first case, assume the evaluation of $S$ completes first, which means the evaluation of $S$ does not depend on the evaluation of $T$. Then the linear system is of the form

[TABLE]

The boldface symbols denote where to pay attention in the linear systems. Perform Gaussian elimination to get

[TABLE]

This corresponds to left-multiplication by the invertible matrix

[TABLE]

As the other case, assume evaluation of $T$ completes first, which means the evaluation of $T$ does not depend on the evaluation of $S$. Then the linear system is of the form

[TABLE]

Perform Gaussian elimination to get

[TABLE]

This corresponds to left-multiplication by the invertible matrix

[TABLE]

Regardless of which of the two cases we start from, we arrive at the same linear system (3). Continue the Gaussian elimination to get

[TABLE]

This corresponds to left-multiplying (3) by the invertible matrix

[TABLE]

We now explicitly parameterize the unspecified parameters one at a time.

[TABLE]

The role of $a$ is to define $z_{1}=az_{0}$. So the evaluation of $(T,S)$ starts with $J_{\alpha A}(az_{0})$. If $a=0$, then $J_{\alpha A}$ ignores the input $z_{0}$ and always uses [math] as the input. Since $(T,S)$ accesses $A$ only through the evaluation of $J_{\alpha A}$, how is it possible that $(T,S)$ evaluates $J_{\alpha A}$ only at [math] and still encodes the zeros of $A+B$?

We now show $a\neq 0$. Assume $a=0$ for contradiction. Let

[TABLE]

where $c_{1}>0$ is unspecified and $0\neq c_{2}\in\mathbb{R}^{d}$. Then $J_{\alpha A}0=0$, and the mappings $T$ and $S$ are independent of the value of $c_{1}$. So the set of fixed points of $T$ and the set of $Sz^{\star}$, where $z^{\star}$ is a fixed point of $T$, do not depend on $c_{1}$. However, the solution $\{-c_{1}^{-1}c_{2}\}=\mathop{\bf zer}(A+B)$ does depend on $c_{1}$. Since $(T,S)$ is assumed to be a fixed-point encoding, $T$ must have a fixed-point $z^{\star}$ and it must satisfy $-c_{1}^{-1}c_{2}=Sz^{\star}$, which is a contradiction.

Knowing $a\neq 0$, we can absorb the top-left $a$ into $z_{0}$ and left-multiply by an invertible matrix to get the equivalent system

[TABLE]

This further simplifies to the equivalent system

[TABLE]

By redefining $(T(z_{0}),S(z_{0}))$ to be the equivalent scaled splitting $(aT(a^{-1}az_{0}),S(a^{-1}az_{0}))$, we get the equivalent system

[TABLE]

where we have now explicitly parameterized the remaining parameters as $\theta_{1},\dots,\theta_{8}$.

The system (4) defines $(T,S)$, i.e., it specifies the evaluation of $(T,S)$