Non-optimality of the Greedy Algorithm for subspace orderings in the method of alternating projections

TL;DR

This paper investigates the limitations of the greedy algorithm for ordering subspaces in the method of alternating projections, showing it is not generally optimal and relating the problem to the NP-hard Traveling Salesman Problem.

Contribution

It proves the non-optimality of the greedy algorithm for subspace ordering and establishes bounds on its performance relative to the optimal solution.

Findings

Greedy algorithm is not generally optimal for subspace ordering.

Optimal ordering problem relates to NP-hard Traveling Salesman Problem.

Constructs Hilbert spaces with prescribed Friedrichs numbers based on arbitrary matrices.

Abstract

The method of alternating projections involves projecting an element of a Hilbert space cyclically onto a collection of closed subspaces. It is known that the resulting sequence always converges in norm and that one can obtain estimates for the rate of convergence in terms of quantities describing the geometric relationship between the subspaces in question, namely their pairwise Friedrichs numbers. We consider the question of how best to order a given collection of subspaces so as to obtain the best estimate on the rate of convergence. We prove, by relating the ordering problem to a variant of the famous Travelling Salesman Problem, that correctness of a natural form of the Greedy Algorithm would imply that $\mathrm{P}=\mathrm{NP}$, before presenting a simple example which shows that, contrary to a claim made in the influential paper [Kayalar-Weinert, Math. Control Signals Systems, vol. 1(1), 1988], the result of the Greedy Algorithm is not in general optimal. We go on to establish sharp estimates on the degree to which the result of the Greedy Algorithm can differ from the optimal result. Underlying all of these results is a construction which shows that for any matrix whose entries satisfy certain natural assumptions it is possible to construct a Hilbert space and a collection of closed subspaces such that the pairwise Friedrichs numbers between the subspaces are given precisely by the entries of that matrix.

Equations

$$\Vert T^{n}x-P_{M}x\Vert \to 0,n\to \infty ,$$

$$c(L_{1},L_{2})=\sup\big{\{}|(x_{1},x_{2})|:x_{k}\in L_{k}\cap L^{\perp}\mbox{ and }\|x_{k}\|\leq 1\mbox{ for }k=1,2\big{\}},$$

$$\Vert T^n-P_M\Vert =c(M_1,M_2{)}^{2n-1},n\ge 1.$$

$$\Vert T^n-P_M\Vert \le c(M_N,M_{N-1}{)}^n\cdots c(M_2,M_1{)}^{n}c(M_1,M_N{)}^{n-1},n\ge 1,$$

$$\Vert T^n-P_M\Vert \le C{r}^n,n\ge 1,$$

$$\Vert T_{\sigma }^n-P_M\Vert \le C_{\sigma }{r}_{\sigma }^n,n\ge 1,$$

$$x_{k,\ell }=\{\begin{array}{ll}{e}_{k,\ell },& 1\le k<\ell \le N,\\ c_{\ell ,k}{e}_{\ell ,k}+(1-c_{\ell ,k}^2{)}^{1/2}{e}_{k,\ell },& 1\le \ell <k\le N.\end{array}$$

$$(x_{k,\ell },x_{k^{\prime }{\ell }^{\prime }})=\{\begin{array}{ll}1,& k=k^{\prime },\ell ={\ell }^{\prime },\\ c_{k,\ell },& k^{\prime }=\ell ,{\ell }^{\prime }=k,\\ 0,& \text{otherwise},\end{array}$$

$$b_{k,\ell }=\{\begin{array}{ll}{c}_{k,\ell },& c_{k,\ell }<1,\\ 0,& c_{k,\ell }=1,\end{array}$$

$$Y_k^{\ell ,m}=\{\begin{array}{ll}U,& c_{\ell ,m}=1\text{ and }k=\ell ,\\ V,& c_{\ell ,m}=1\text{ and }k=m,\\ \{0\},& \text{otherwise},\end{array}$$

$$w\colon\big{\{}(k,\ell)\in V_{N}^{2}:k\neq\ell\big{\}}\to\mathbb{R}$$

$$\Sigma_{\sigma}=\sum_{k=1}^{N}w\big{(}\sigma(k),\sigma(k+1)\big{)}$$

$$\Pi_{\sigma}=\prod_{k=1}^{N}w\big{(}\sigma(k),\sigma(k+1)\big{)}.$$

$$x_1$$

$$x_2$$

$$x_3$$

$$x_4$$

$$C=\left(\begin{array}[]{cccc}0.0000&0.5000&0.2000&0.1000\\ 0.5000&0.0000&0.1612&0.1583\\ 0.2000&0.1612&0.0000&0.0940\\ 0.1000&0.1583&0.0940&0.0000\end{array}\right).$$

$$\big{(}\sigma_{G}(k)\big{)}_{k=1}^{4}=(1,4,3,2),$$

$$\big{(}\sigma(k)\big{)}_{k=1}^{4}=(1,4,2,3)$$

$$r_{\ast }\le r_G\le r_{\ast }^{1/2}.$$

$$s_{k}(\ell)=\sigma_{k}\big{(}\sigma_{k}^{-1}(\ell)+1\big{)}$$

$$w\big{(}\sigma(\ell),s_{k}(\sigma(\ell)\big{)}\leq w\big{(}\sigma(\ell),\sigma(\ell+1)\big{)},$$

$$w\big{(}\sigma(\ell+1),s_{k}(\sigma(\ell+1)\big{)}\leq w\big{(}\sigma(\ell),\sigma(\ell+1)\big{)}.$$

$$w\big{(}\sigma(\ell),s_{k}(\sigma(\ell)\big{)}w\big{(}\sigma(\ell+1),s_{k}(\sigma(\ell+1)\big{)}\leq w\big{(}\sigma(\ell),\sigma(\ell+1)\big{)}$$

$$r_k^2$$

$$\displaystyle=\prod_{\ell=1}^{N}\Big{(}w\big{(}\sigma(\ell),s_{k}(\sigma(\ell)\big{)}w\big{(}\sigma(\ell+1),s_{k}(\sigma(\ell+1)\big{)}\Big{)}$$

$$\displaystyle\leq\prod_{\ell=1}^{N}w\big{(}\sigma(\ell),\sigma(\ell+1)\big{)}=\prod_{\ell=1}^{N}c(M_{\sigma(\ell)},M_{\sigma(\ell+1)})=r_{\sigma}.$$

$$w\big{(}\sigma_{*}(\ell),\sigma_{*}(\ell+1)\big{)}=\min\big{\{}w(\sigma_{*}(\ell),k):1\leq k\leq N,\,k\neq\sigma_{*}(\ell)\big{\}}$$

$$r_G>(1-\epsilon )r_{\ast }^{1/2}.$$

$$c(M_k,M_{\ell })=\{\begin{array}{ll}c& \text{if k=ℓ±1(modN)}\\ c\delta & \text{if k=ℓ±2(modN) and k is even,}\\ 1& \text{otherwise}.\end{array}$$

$$r_{\sigma }\ge c^{n-1}(c\delta {)}^{n-1}=c^{N-2}{\delta }^{n-1}.$$

$$r_G\ge c^2(c\delta {)}^{n-1}=c{\delta }^{n-1}{r}_{\ast }^{1/2}.$$

Full text

1. Introduction

Let $X$ be a real or complex Hilbert space, $N\geq 2$ an integer, and suppose that $M_{1},\dotsc,M_{N}$ are closed subspaces of $X$. Furthermore let $P_{k}$ denote the orthogonal projection onto $M_{k}$, $1\leq k\leq N$, and let $P_{M}$ denote the orthogonal projection onto the intersection $M=M_{1}\cap\dotsc\cap M_{N}$. If we let $T=P_{N}\cdots P_{1}$ then it follows from a classical theorem due to Halperin [8] that

[TABLE]

for all $x\in X$. It follows easily that, for any $x\in X$, the sequence in $X$ obtained by starting at $X$ and then projecting cyclically onto the $N$ subspaces $M_{1},\dotsc,M_{N}$ must converge to the point $P_{M}x$, which is the point in $M$ closest to the starting vector $x$. This procedure is known as the method of alternating projections and has many applications, for instance to the iterative solution of large linear systems but also in the theory of partial differential equations and in image restoration; see [3] for a survey.

In view of these applications it is important to understand the rate at which the convergence in (1.1) takes place; see for instance [1, 2, 6, 7] for in-depth investigations. Recall that the Friedrichs number $c(L_{1},L_{2})$ between the two subspaces $L_{1},L_{2}$ of $X$ is defined as

[TABLE]

where $L=L_{1}\cap L_{2}$. The Friedrichs number lies in the interval $[0,1]$ and may be thought of as the cosine of the ‘angle’ between the subspaces $L_{1}$ and $L_{2}$. It is shown in [9, Theorem 2] that for $N=2$ in the method of alternating projections we have

[TABLE]

When $N\geq 3$ no sharp upper bound of this form is known, but it is shown in [5, Corollary 2.10] that

[TABLE]

provided the subspaces are pairwise quasi-disjoint in the sense that $M_{k}\cap M_{\ell}\cap M^{\perp}=\{0\}$ for $1\leq k,\ell\leq N$ with $k\neq\ell$. Moreover, the assumption on the subspaces cannot be omitted. The same bound was obtained earlier in [9] in the special case where the subspaces $M_{1}\cap M^{\perp},\dots,M_{N}\cap M^{\perp}$ are independent, which is to say that if vectors $x_{k}\in M_{k}\cap M^{\perp}$, $1\leq k\leq N$, satisfy $x_{1}+\dots+x_{N}=0$ then $x_{1}=\dotsc=x_{N}=0$.

Examples in [5, Section 3] show both that the bound in (1.4) fails to be sharp in some special cases, thus disproving a conjecture made in [9], and more generally that it is not possible for $N\geq 3$ to obtain a sharp upper bound for $\|T^{n}-P_{M}\|$, $n\geq 1$, which depends only on the pairwise Friedrichs numbers between the subspaces $M_{1},\dots,M_{N}$. Nevertheless, the estimate in (1.3) recovers the sharp bound in (1.2) when $N=2$ and holds with equality in a number of other cases, for instance if all of the spaces $M_{1},\dotsc,M_{N}$ are one-dimensional. We also see from (1.3) that if the Friedrichs number between a pair of consecutive subspaces is zero then we have convergence in the method of alternating projections after at most two steps. Since our interest here is primarily in the asymptotic rate of convergence as $n\to\infty$, there is no significant loss of generality in assuming that $c(M_{k},M_{\ell})>0$ for $1\leq k,\ell\leq N$ with $k\neq\ell$. In this case (1.3) may be recast as

[TABLE]

where $C=c(M_{1},M_{N})^{-1}$ and $r=\prod_{k=1}^{N}c(M_{k},M_{k+1})$, indices henceforth being considered modulo $N$. Since the asymptotic rate of convergence is determined by the value of $r\in(0,1]$, it is natural to seek the reordering of the subspaces $M_{1},\dots,M_{N}$ which leads to the smallest possible value of $r$. More formally, given $N\geq 2$ we let $S_{N}$ denote the symmetric group on $N$ letters and for each $\sigma\in S_{N}$ we let $r_{\sigma}=\prod_{k=1}^{N}c(M_{\sigma(k)},M_{\sigma(k+1)}),$ so that for the reordered product $T_{\sigma}=P_{\sigma(N)}\cdots P_{\sigma(1)}$ we obtain

[TABLE]

where $C_{\sigma}=c(M_{\sigma(1)},M_{\sigma(N)})^{-1}$. The objective therefore is to find a permutation $\sigma\in S_{N}$ such that $r_{\sigma}=r_{*}$, where $r_{*}=\min\{r_{\sigma}:\sigma\in S_{N}\}$, and to find such a permutation a version of the following ‘greedy’ algorithm was proposed in [9, Section 9].

Greedy Algorithm: Given $N\geq 2$ independent closed subspaces $M_{1},\dots,M_{N}$ of a Hilbert space $X$ whose mutual Friedrichs numbers are known we obtain permutations $\sigma_{k}\in S_{N}$, $1\leq k\leq N$, as follows. Let $\sigma_{k}(1)=k$ and for $j=2,\dots,N$ consider as possible values for $\sigma_{k}(j)$ any previously unused index $\ell$ which minimises $c(M_{\sigma_{k}(j-1)},M_{\ell})$. If at any stage there is more than one choice of such an index $\ell$ then proceed by considering all possible choices of this index and take $\sigma_{k}$ to be that permutation which among those leading to the least value of $r_{\sigma_{k}}$ comes first in the lexicographical ordering. Return the permutation $\sigma_{G}=\sigma_{\ell}$ where $\ell\in\{1,\dots,N\}$ is the smallest index such that $r_{\sigma_{\ell}}=\min\{r_{\sigma_{k}}:1\leq k\leq N\}$.

If we let $r_{G}=r_{\sigma_{G}}$, $N\geq 2$, then the Greedy Algorithm is correct if and only if $r_{G}=r_{*}$ for all constellations of subspaces. By definition of $r_{*}$ it is clear that $r_{*}\leq r_{G}$, $N\geq 2$. In Section 3 we show that if the Greedy Algorithm were correct then it would follow that $\mathrm{P}=\mathrm{NP}$. We then exhibit a simple example with $N=4$ in which $r_{*}<r_{G}$. Both results are obtained as a consequence of a construction, presented in Section 2, which shows that any suitable collection of numbers in $[0,1]$ arises as the set of pairwise Friedrichs numbers between subspaces of some Hilbert space. This result is of independent interest and in particular implies that the problem of finding an optimal ordering is at least as hard as solving a multiplicative form of the Travelling Salesman Problem (TSP). In Section 4 we give sharp estimates for the maximal discrepancies between $r_{*}$ and $r_{G}$. In particular, we show that generically $r_{G}<\smash{r_{*}^{1/2}}$, and that the estimate is optimal in the sense that for every $\varepsilon\in(0,1)$ there exists some $N\geq 2$ and a suitable collection of $N$ subspaces of some Hilbert space such that $r_{G}>(1-\varepsilon)\smash{r_{*}^{1/2}}.$ The last step once again requires the construction from Section 2.

2. Friedrichs matrices

Given $N\geq 2$ closed subspaces $M_{1},\dots,M_{N}$ of a Hilbert space, we may consider the $N\times N$-matrix $(c(M_{k},M_{\ell}))_{1\leq k,\ell\leq N}$ whose entries are the pairwise Friedrichs numbers between the various subspaces. We call the matrix arising in this way the Friedrichs matrix corresponding to the collection of subspaces. It is clear that any Friedrichs matrix must be symmetric, have zeros along its main diagonal and elsewhere must have entries lying in the interval $[0,1]$. Is every square matrix which has these three properties a Friedrichs matrix for some collection of closed subspaces? The following result answers this question in the affirmative. Here and in what follows we use the same notation as in Section 1.

Theorem 2.1**.**

Let $\mathbb{F}\in\{\mathbb{R},\mathbb{C}\}$ and $N\geq 2$, and suppose that $C$ is an $N\times N$-matrix which is symmetric, has zeros along its main diagonal and elsewhere has entries lying in the interval $[0,1]$. Then there exists a Hilbert space $X$ over the field $\mathbb{F}$ and closed subspaces $M_{1},\dots,M_{N}$ of $X$ such that $C$ is the corresponding Friedrichs matrix. Furthermore, the subspaces can be constructed in such a way that $M_{k}\cap M_{\ell}=\{0\}$ for $1\leq k,\ell\leq N$ with $k\neq\ell$ and, if $N\geq 3$, $P_{k}P_{\ell}P_{m}=0$ for $1\leq k,\ell,m\leq N$ mutually distinct.

Proof.

Let $C=(c_{k,\ell})$ and suppose first that $0\leq c_{k,\ell}<1$ for $1\leq k,\ell\leq N$. Let $\{e_{k,\ell}:1\leq k,\ell\leq N,k\neq\ell\}$ be an orthonormal basis for the space $X=\mathbb{F}^{N(N-1)}$ endowed with the Euclidean norm, and set

[TABLE]

For $1\leq k\leq N$ let $B_{k}=\{x_{k,\ell}:1\leq\ell\leq N,\,\ell\neq k\}$, noting that these sets are orthonormal, and consider the closed subspaces of $X$ given by $M_{k}=\operatorname*{span}B_{k}$. By our assumption that the entries of $C$ be strictly smaller than 1 we see that $M_{k}\cap M_{\ell}=\{0\}$ for $1\leq k,\ell\leq N$ with $k\neq\ell$, and in particular $M=\{0\}$. Furthermore, for $1\leq k,k^{\prime},\ell,\ell^{\prime}\leq N$ with $k\neq\ell$ and $k^{\prime}\neq\ell^{\prime}$ we have

[TABLE]

from which it follows that $c(M_{k},M_{\ell})=c_{k,\ell}$ for $1\leq k,\ell\leq N$ with $k\neq\ell$ and, if $N\geq 3$, that $P_{k}P_{\ell}P_{m}=0$ for $1\leq k,\ell,m\leq N$ mutually distinct.

Now consider the general case where $0\leq c_{k,\ell}\leq 1$ for $1\leq k,\ell\leq N$ and consider the matrix $B=(b_{k,\ell})$ with entries

[TABLE]

for $1\leq k,\ell\leq N$. By the first part we may find closed subspaces $L_{1},\dotsc,L_{N}$ of $\mathbb{F}^{N(N-1)}$ whose Friedrichs matrix is $B$. Let $X=\mathbb{F}^{N(N-1)}\oplus Y$, where $Y=\bigoplus_{1\leq\ell<m\leq N}\ell^{2}$, and endow $X$ with its natural Hilbert space norm. Moreover, let $U,V$ be two closed subspaces of $\ell^{2}$ such that $U+V$ is not closed. For $1\leq k,\ell,m\leq N$ with $\ell<m$ define the subspaces $Y^{\ell,m}_{k}$ of $\ell^{2}$ by

[TABLE]

and for $1\leq k\leq N$ define the closed subspace $M_{k}$ of $X$ by $M_{k}=L_{k}\oplus Y_{k}$, where $Y_{k}=\bigoplus_{1\leq\ell<m\leq N}Y^{\ell,m}_{k}$. If $1\leq k<\ell\leq N$ are such that $c_{k,\ell}<1$, then for $1\leq m<n\leq N$ we have either $Y_{k}^{m,n}=\{0\}$ or $Y_{\ell}^{m,n}=\{0\}$ and therefore $c(M_{k},M_{\ell})=c(L_{k},L_{\ell})=b_{k,\ell}=c_{k,\ell}$. Suppose that $1\leq k<\ell\leq N$ and that $c_{k,\ell}=1$. Then for $1\leq m<n\leq N$ we see that $Y^{m,n}_{k}+Y_{\ell}^{m,n}=U+V$ if and only if $k=m$ and $\ell=n$, and that otherwise $Y^{m,n}_{k}+Y_{\ell}^{m,n}$ equals either $U,V$ or $\{0\}$. It follows that $Y_{k}+Y_{\ell}$ is not closed, and hence $M_{k}+M_{\ell}$ is not closed. By [4, Theorem 9.35] this implies that $c(M_{k},M_{\ell})=1=c_{k,\ell}$, and hence we have the required subspaces. Moreover, it is clear from the construction that $M_{k}\cap M_{\ell}=\{0\}$ for $1\leq k,\ell\leq N$ with $k\neq\ell$ and, if $N\geq 3$, that $P_{k}P_{\ell}P_{m}=0$ for $1\leq k,\ell,m\leq N$ mutually distinct. ∎

Remark 2.2**.**

Note that the result in particular provides a new proof of the fact that in general the optimal value of $r$ in (1.4) cannot be expressed as a function of pairwise Friedrichs numbers between the subspaces $M_{1},\dots,M_{N}$ when $N\geq 3$, as was first observed in a particular case in [5, Example 3.7]. Indeed, for any collection of closed subspaces $M_{1}^{\prime},\dots,M_{N}^{\prime}$, $N\geq 3$, of some Hilbert space such that in the method of alternating projections we do not have convergence in one step, by Theorem 2.1 we may find an alternative collection of closed subspaces $M_{1},\dots,M_{N}$ of some Hilbert space with the same pairwise Friedrichs numbers but for which $T=P_{M}=0$.

3. Incorrectness of the Greedy Algorithm

In this section we turn to the Greedy Algorithm presented in Section 1, and in particular we ask whether the algorithm is correct in the sense that the ordering it produces leads to the optimal value of $r\in[0,1]$ in (1.4). We first consider the connection between our problem of finding an optimal ordering and the classical TSP, and we show in Corollary 3.3 below that correctness of the Greedy Algorithm for a sufficiently large class of cases would imply that $\mathrm{P}=\mathrm{NP}$. We then exhibit a simple example in which the Greedy Algorithm gives a suboptimal ordering.

Recall that in the graph-theoretical formulation of the TSP we are given, for some $N\geq 2$, a complete graph $K_{N}$ with vertices $V_{N}=\{1,2,\dotsc,N\}$ and a weight function

[TABLE]

such that $w(k,\ell)=w(\ell,k)$ for $1\leq k,\ell\leq N$ with $k\neq\ell$, and the objective is to find a permutation $\sigma^{*}\in S_{N}$ such that $\Sigma_{\sigma^{*}}=\min\{\Sigma_{\sigma}:\sigma\in S_{N}\}$, where for a permutation $\sigma\in S_{N}$ we let

[TABLE]

with indices, as usual, considered modulo $N$. We will be interested primarily in the multiplicative form of the TSP, denoted by MTSP, in which the objective is to minimise not the additive cost but instead to find $\sigma^{*}\in S_{N}$ such that $\Pi_{\sigma^{*}}=\min\{\Pi_{\sigma}:\sigma\in S_{N}\}$, where for a permutation $\sigma\in S_{N}$ we let

[TABLE]

It is clear that TSP and MTSP have the same solution, and indeed one may pass from one form of the problem to the other simply by replacing the weight function by its logarithm or its exponential, as appropriate. Furthermore, the solution of TSP is unaffected by shifting the values of the weight function by a constant amount, which implies in particular that there is no loss of generality in considering the MTSP only for weight functions taking values in the range $[0,1]$.

It is well known that the TSP, and hence also MTSP, is NP-complete. This means that it lies in the complexity class NP and is NP-hard, which is to say that any other problem in NP can be transformed into an instance of the TSP in polynomial time. Furthermore, by considering the corresponding decision problems it can be seen that TSP and hence MTSP remain NP-complete if the weight function is assumed to take distinct values on distinct pairs. Our first result is an application of Theorem 2.1 showing that the subspace ordering problem is NP-hard.

Proposition 3.1**.**

The problem of finding an optimal ordering for collections of independent closed subspaces with pairwise distinct Friedrichs numbers is NP-hard.

Proof.

It suffices to show that every instance of TSP with distinct costs can be transformed in polynomial time into a subspace ordering problem with pairwise distinct Friedrichs numbers. However, this follows straightforwardly from Theorem 2.1. Indeed, given a TSP problem on $N\geq 2$ vertices we may transform it to an instance of MTSP with weight function taking values in the range $[0,1]$ in $O(N^{2})$ steps. Let $C=(c_{k,\ell})_{1\leq k,\ell\leq N}$ be the symmetric matrix with zeros along its main diagonal and entries $c_{k,\ell}=w(k,\ell)$ for $1\leq k,\ell\leq N$ with $k\neq\ell$. By Theorem 2.1 there exists a Hilbert space $X$ and independent closed subspaces $M_{1},\dots,M_{N}$ of $X$ such that $C$ is the associated Friedrichs matrix. Moreover, it is clear from the proof of Theorem 2.1 that it is possible to obtain these subspaces in polynomial time. If we find a permutation $\sigma^{*}\in S_{N}$ such that $r_{\sigma^{*}}=r_{*}$, then since $r_{\sigma}=\Pi_{\sigma}$ for all $\sigma\in S_{N}$ the permutation $\sigma^{*}$ also solves our instance of MTSP, and hence the original TSP problem. Since TSP is known to be NP-hard, our problem is too. ∎

Remark 3.2**.**

Note that the subspaces $M_{1},\dots,M_{N}$ are not merely independent but satisfy the much stronger conditions described in Theorem 2.1. In particular, the result remains true if the subspaces which we are trying to order are merely pairwise quasi-disjoint in the sense of Section 1.

The result shows that the existence of any polynomial-time algorithm which solves the subspace ordering problem in a sufficiently large number of cases implies that $\mathrm{P}=\mathrm{NP}$. In particular, we obtain the following consequence for the Greedy Algorithm.

Corollary 3.3**.**

Correctness of the Greedy Algorithm for independent subspaces with pairwise distinct Friedrichs numbers implies that $\mathrm{P}=\mathrm{NP}$.

Proof.

It is straightforward to see that if all the pairwise Friedrichs numbers are distinct then the Greedy Algorithm terminates after $O(N^{3})$ steps, where $N\geq 2$ is the number of subspaces we a required to order optimally. ∎

Remark 3.4**.**

The version of the Greedy Algorithm formulated in [9, Section 9] differs from ours in that it does not consider all possible greedy paths and hence runs in polynomial time even if the pairwise Friedrichs numbers are not assumed to be distinct. Note also that, as in the case of Proposition 3.1, the assumption of independence on the subspaces can be relaxed to pairwise quasi-disjointness.

Given that the question whether $\mathrm{P}=\mathrm{NP}$ is a long-standing open problem, one may view Proposition 3.1 as evidence suggesting that the Greedy Algorithm does not in general lead to an optimal ordering of the subspaces in question. This is indeed the case, as the following example illustrates.

Example 3.5**.**

Let $\mathbb{F}\in\{\mathbb{R},\mathbb{C}\}$ and let $X=\mathbb{F}^{4}$ with the Euclidean norm. Consider the one-dimensional subspaces $M_{k}=\operatorname*{span}\{x_{k}\}$, $1\leq k\leq 4$, where $x_{1},\dotsc,x_{4}\in X$ are the unit vectors

[TABLE]

The Friedrichs numbers satisfy $c(M_{k},M_{\ell})=|(x_{k},x_{\ell})|$ for $1\leq k,\ell\leq 4$ with $k\neq\ell$, so the associated Friedrichs matrix is given (approximately) by

[TABLE]

The permutation $\sigma_{G}\in S_{4}$ produced by the Greedy Algorithm is

[TABLE]

which leads to $r_{G}\approx 7.5772\times 10^{-4}$. The permutation $\sigma\in S_{4}$ given by

[TABLE]

leads to the optimal value $r_{*}=r_{\sigma}\approx 5.1033\times 10^{-4}$, and in particular $r_{G}>r_{*}$. It follows that the Greedy Algorithm is not correct.

Remark 3.6**.**

Example 3.5 disproves a claim made in [9, Section 9], namely that the Greedy Algorithm always leads to an optimal ordering in the case of independent subspaces. The examples considered in [9, Section 9] involve only $N=3$ subspaces, a special case in which the Greedy Algorithm performs an exhaustive search of all possible orderings (up to the direction in which they are traversed) and in particular is correct. Thus Example 3.5 is minimal in terms of the number of subspaces involved.

4. Sharp estimates for the degree of suboptimality

Having shown in Section 3 that the Greedy Algorithm does not in general lead to an optimal ordering of the subspaces in the method of alternating projections, we seek now to quantify how much the result reached by the Greedy Algorithm can disagree with the optimal result. Given a collection of closed subspaces of a Hilbert space such that at least one of the pairwise Friedrichs numbers is zero, we see that for suitable orderings of the subspaces we obtain convergence after at most two steps in the method of alternating projections. Another essentially uninteresting case for asymptotic analysis is when all of the pairwise Friedrichs numbers equal 1, so that no ordering leads to a useful estimate in (1.3). If either of these two cases holds we shall say that the collection of subspaces involved is non-generic, and otherwise we call it generic.

Theorem 4.1**.**

Let $N\geq 2$ and suppose that $M_{1},\dotsc,M_{N}$ are closed subspaces of a Hilbert space $X$. Then

[TABLE]

Moreover, the second inequality is strict unless the collection $M_{1},\dotsc,M_{N}$ of subspaces is non-generic

Proof.

For $1\leq k\leq N$ let $\sigma_{k}\in S_{N}$ be the permutation produced by running the Greedy Algorithm with the starting vertex $\sigma_{k}(1)=k$ and let $r_{k}=r_{\sigma_{k}}$. Then certainly $r_{*}\leq r_{k}$ for $1\leq k\leq N$, and hence also $r_{*}\leq r_{G}$. For $1\leq k,\ell\leq N$ let

[TABLE]

denote the index of the successor to $M_{\ell}$ in the ordering of the subspaces determined by $\sigma_{k}$, noting that $s_{k}(\ell)=1$ if $\sigma_{k}(\ell)=N$. Let $\sigma\in S_{N}$ and for $1\leq k,\ell\leq N$ with $k\neq\ell$ let $w(k,\ell)=c(M_{k},M_{\ell})$. Let $1\leq k,\ell\leq N$. If $\sigma_{k}^{-1}(\sigma(\ell))<\sigma_{k}^{-1}(\sigma(\ell+1))$, which is to say that in the ordering determined by $\sigma_{k}$ the subspace $M_{\sigma(\ell)}$ comes before $M_{\sigma(\ell+1)}$, then by definition of the Greedy Algorithm we must have

[TABLE]

while if $\sigma_{k}^{-1}(\sigma(\ell))>\sigma_{k}^{-1}(\sigma(\ell+1))$ then

[TABLE]

Since $w$ takes values in $[0,1]$ it follows that

[TABLE]

for $1\leq k,\ell\leq N$. Thus for $1\leq k\leq N$ we have

[TABLE]

Since $\sigma\in S_{N}$ was arbitrary we deduce that $r_{k}^{2}\leq r_{*}$ for $1\leq k\leq N$, and in particular $r_{G}^{2}\leq r_{*}$, as required.

Now suppose that $r_{G}^{2}=r_{*}$, and let $\sigma_{*}\in S_{N}$ be a permutation such that $r_{\sigma_{*}}=r_{*}$. Since $r_{G}^{2}\leq r_{k}^{2}\leq r_{*}$ for $1\leq k\leq N$, we see that in fact $r_{k}^{2}=r_{*}$ for $1\leq k\leq N$. Now either one of the pairwise Friedrichs numbers is zero or all of the pairwise Friedrichs numbers are non-zero. In the latter case it is clear from (4.3) that we must have equality in (4.2) for $1\leq k,\ell\leq N$ when $\sigma=\sigma_{*}$. Taking $k=\sigma_{*}(\ell)$ in (4.2) for $1\leq\ell\leq N$, it follows that

[TABLE]

for $1\leq\ell\leq N$. It follows that $\sigma_{*}$ is itself a permutation considered by the Greedy Algorithm, and therefore $r_{*}=r_{G}$. Hence $r_{*}^{2}=r_{*}$, and since $r_{*}\neq 0$ we have $r_{*}=1$, which implies that $c(M_{k},M_{\ell})=1$ for $1\leq k,\ell\leq N$ with $k\neq\ell$. It follows that $r_{G}^{2}<r_{*}$ unless the collection $M_{1},\dotsc,M_{N}$ of subspaces is non-generic. ∎

It remains to be investigated to what extent the second bound in (4.1) is sharp for generic constellations of subspaces. Our final example shows that it cannot be improved in the sense that given any $\varepsilon\in(0,1)$ there exists a generic constellation of subspaces of some Hilbert space such that

[TABLE]

In fact, there exists a constellation of $N$ such subspaces for every even $N\geq 4$.

Example 4.2**.**

Given a positive a positive integer $n\geq 2$, let $N=2n$ and suppose that $0<\delta<c<1$. By Theorem 2.1 there exists a Hilbert space $X$ and a generic constellation $M_{1},\dotsc,M_{N}$ of closed subspaces of $X$ such that for $1\leq k,\ell\leq N$ with $k\neq\ell$ we have

[TABLE]

Let $\sigma_{0}\in S_{N}$ denote the identity permutation. Then $r_{\sigma_{0}}=c^{N}$. If we think of the subspaces as the vertices of a complete graph of order $N$, and we let the edges have weights given by the pairwise Friedrichs numbers, then $r_{\sigma}\geq r_{\sigma_{0}}$ for all permutations $\sigma\in S_{N}$ involving no $c\delta$-edges. Moreover, any cycle $\sigma\in S_{N}$ which uses at least one of the $c\delta$-edges cannot use more than $n-1$ of them, and must involve at least two 1-edges, so for any such cycle

[TABLE]

In particular, if $c^{2}\leq\delta^{n-1}$ then $r_{*}=r_{\sigma_{0}}$. It is easy to that

[TABLE]

Given $\varepsilon\in(0,1)$ we deduce that $r_{G}>(1-\varepsilon)r_{*}^{1/2}$ provided $c,\delta\in(0,1)$ are such that $c^{2}\leq\delta^{n-1}$ and $c\delta^{n-1}>1-\varepsilon$. These conditions are satisfied for instance when $(1-\varepsilon)^{1/3}<c<1$ and $\delta=c^{2/(n-1)}$. Furthermore, it is the case that for any $r,\varepsilon\in(0,1)$ there exist generic constellations of $N$ subspaces of a Hilbert space for all sufficiently large even $N\geq 4$ with the properties that $r_{G}>\smash{(1-\varepsilon)r_{*}^{1/2}}$ and $r_{*}=r$.

5. Acknowledgements

For financial support O.D. thanks Magdalen College, Oxford, A.J. thanks the Mathematical Institute of the University of Oxford, S.R. and R.S. thank both St John’s College, Oxford, and the Mathematical Institute, and L.S. thanks the EPSRC. All authors would further like to express their thanks to Alexis Chevalier, Stefan Kiefer, Dominik Peters and Zhixuan Wang for useful discussions.