On the convex geometry of blind deconvolution and matrix completion

TL;DR

This paper introduces a geometric approach to analyze low-rank matrix recovery problems like matrix completion and blind deconvolution, revealing intrinsic conditioning issues and providing near-optimal error bounds under realistic noise conditions.

Contribution

It offers a novel geometric perspective that uncovers the intrinsic conditioning limitations of these problems and derives near-optimal error estimates for blind deconvolution with adversarial noise.

Findings

Dimension factors are due to intrinsic problem conditioning.

Bad conditioning occurs only at very small noise levels.

Near-optimal error bounds are achieved under realistic noise assumptions.

Abstract

Low-rank matrix recovery from structured measurements has been a topic of intense study in the last decade and many important problems like matrix completion and blind deconvolution have been formulated in this framework. An important benchmark method to solve these problems is to minimize the nuclear norm, a convex proxy for the rank. A common approach to establish recovery guarantees for this convex program relies on the construction of a so-called approximate dual certificate. However, this approach provides only limited insight in various respects. Most prominently, the noise bounds exhibit seemingly suboptimal dimension factors. In this paper we take a novel, more geometric viewpoint to analyze both the matrix completion and the blind deconvolution scenario. We find that for both these applications the dimension factors in the noise bounds are not an artifact of the proof, but the problems are intrinsically badly conditioned. We show, however, that bad conditioning only arises for very small noise levels: Under mild assumptions that include many realistic noise levels we derive near-optimal error estimates for blind deconvolution under adversarial noise.

Equations

$$y=\mathcal{A}\left(X_0\right)+e\in {\mathbb{C}}^m,$$

$$\Vert X{\Vert }_{\ast }$$

$$\Vert y-\mathcal{A}\left(X\right)\Vert \le \tau ,$$

$$\Vert \hat{X}-X_0{\Vert }_F\lesssim \sqrt{n_1}\tau ,$$

$$\Vert \hat{X}-X_0{\Vert }_F\lesssim \tau .$$

$${\lambda }_{\min }\left(\mathcal{A},\mathcal{K}\right):=\underset{Z\in \mathcal{K}\backslash \left\{0\right\}}{\inf }\frac{\Vert \mathcal{A}\left(Z\right)\Vert }{\Vert Z{\Vert }_F}$$

$$w\ast x:={\left(\sum _{j=1}^L{w}_j{x}_{k-j}\right)}_{k=1}^L,$$

$$w\ast x=\sqrt{L}{F}^{\ast }\text{diag}\left(Fw\right)Fx,$$

$$\widehat{w\ast x}:=F\left(w\ast x\right)=\sqrt{L}\text{diag}\left(FB{h}_0\right)FC{\overline{m}}_0.$$

$${\left(\widehat{w\ast x}\right)}_{\ell }=b_{\ell }^{\ast }{h}_0{m}_0^{\ast }{c}_{\ell }=\text{Tr}\left(h_0{m}_0^{\ast }{c}_{\ell }{b}_{\ell }^{\ast }\right)=⟨b_{\ell }{c}_{\ell }^{\ast },h_0{m}_0^{\ast }{⟩}_F.$$

$$\left(\mathcal{A}\left(X\right)\right)\left(\ell \right):=⟨b_{\ell }{c}_{\ell }^{\ast },X{⟩}_F$$

$$y=\widehat{w\ast x}+e=\mathcal{A}\left(X_0\right)+e,$$

$$\begin{array}{rl}\text{minimize }& \Vert X{\Vert }_{\ast }\\ \text{subject to}& \Vert \mathcal{A}\left(X\right)-y\Vert \le \tau .\end{array}$$

$$L/{\log }^{3}L\gtrsim K{\mu }_{\max }^2+N\max \left\{{\mu }_{h_0}^2;{\tilde{\mu }}_{h_0}^2\right\}.$$

$$\Vert \hat{X}-h_0{m}_0^{\ast }{\Vert }_F\lesssim \sqrt{K+N}\tau .$$

$${\mu }_{\max }^2:=\frac{L}{K}\underset{\ell \in \left[L\right]}{\max }\Vert b_{\ell }{\Vert }^2.$$

$${\mu }_{h_0}^2:=\frac{L}{\Vert h_0{\Vert }^2}\underset{\ell \in \left[L\right]}{\max }|⟨b_{\ell },h_0⟩{|}^2.$$

$$\mathcal{A}\left(X\right)\left(i\right):=\sqrt{\frac{n_1{n}_2}{m}}⟨X,e_{a_i}{e}_{b_i}^{\ast }{⟩}_F,$$

$$\begin{array}{rl}\text{minimize }& \Vert X{\Vert }_{\ast }\\ \text{subject to}& \Vert \mathcal{A}\left(X\right)-y\Vert \le \tau .\end{array}$$

$$\mu \left(U\right)$$

$$\mu \left(V\right)$$

$$m\ge C\max \left\{{\mu }^2\left(U\right);{\mu }^2\left(V\right)\right\}r{n}_1\log \left(n_1\right)\log \left[r\left(\mu \left(U\right)+\mu \left(V\right)\right)\right].$$

$$\Vert \hat{X}-X_0{\Vert }_F\lesssim \tau \sqrt{n_2},$$

$$\Vert \hat{X}-X_0\Vert \lesssim {\kappa }^2\sqrt{rm}\Vert e{\Vert }_{\infty },$$

$${\mathcal{K}}_{\ast }\left(X_0\right):=\left\{Z\in {\mathbb{C}}^{K\times N}:\Vert X_0+\epsilon Z{\Vert }_{\ast }\le \Vert X_0{\Vert }_{\ast }\text{for some }\epsilon >0\right\}.$$

$${\lambda }_{\min }\left(\mathcal{A},{\mathcal{K}}_{\ast }\left(X_0\right)\right):=\underset{Z\in {\mathcal{K}}_{\ast }\left(X_0\right)\setminus \left\{0\right\}}{\inf }\frac{\Vert \mathcal{A}\left(Z\right)\Vert }{\Vert Z{\Vert }_F}.$$

$$\Vert \hat{X}-X_0{\Vert }_F\le \frac{2\tau }{{\lambda }_{\min }\left(\mathcal{A},{\mathcal{K}}_{\ast }\left(X_0\right)\right)}.$$

$$C_{1}K\le L\le \frac{KN}{36}.$$

$$\Vert \tilde{X}-X_0{\Vert }_F\ge \frac{\tau }{C_3}\sqrt{\frac{KN}{L}}.$$

$$C_{1}K\le L\le \frac{KN}{36}.$$

$${\lambda }_{\min }\left(\mathcal{A},{\mathcal{K}}_{\ast }\left(h_0{m}_0^{\ast }\right)\right)\le C_3\sqrt{\frac{L}{KN}}.$$

Full text

1 Introduction

A number of recent works have explored the observation that various ill-posed inverse problems in signal processing, imaging, and machine learning can be naturally formulated as the task of recovering a low-rank matrix $X_{0}\in\mathbb{C}^{n_{1}\times n_{2}}$ from an underdetermined system of structured linear measurements

[TABLE]

where $\mathcal{A}:\mathbb{C}^{n_{1}\times n_{2}}\rightarrow\mathbb{C}^{m}$ is a linear map and $e\in\mathbb{C}^{m}$, $\|e\|\leq\tau$, represents additive noise. Such problems include, for example, matrix completion [8], phase retrieval [9], blind deconvolution [1], robust PCA [6], and demixing [49]. In this paper, we aim to analyze the worst case scenario, that is, we do not make any assumptions on the noise except for the bound on its Euclidean norm (this scenario is sometimes referred to as adversarial noise, as it allows for noise specifically designed to be most harmful in a given situation). A natural first approach to recover $X_{0}$ that remains an important benchmark is to solve the semidefinite program

[TABLE]

where $\|\cdot\|_{\ast}$ denotes the nuclear norm, i.e., the sum of the singular values. Recovery guarantees have been shown under the assumption that the measurement operator $\mathcal{A}$ possesses a certain degree of randomness. To establish such guarantees various proof strategies have been proposed, including approaches via the restricted isometry property [55, 47], descent cone analysis [13], and so-called approximate dual certificates [26, 25]. While the latter approach remains state of the art for many structured problems including the highly relevant problems of randomized blind deconvolution and matrix completion, it seemingly has some disadvantages. Most prominently, the resulting recovery guarantees take the form

[TABLE]

where $\hat{X}$ denotes a minimizer of the semidefinite program above and $\|\cdot\|_{F}$ denotes the Frobenius norm, whereas under comparable normalization, the first two approaches, when applicable, give rise to superior recovery guarantees of the form

[TABLE]

Before this paper it was open whether the additional dimension scaling factor in (1) is a proof artifact. Similarly, for randomized blind deconvolution one of the coherence terms appearing in the result was believed to arise only from the proof technique (cf. [46, Remark 2]).

Another drawback of proceeding via an approximate dual certificate is that it gives only limited insight into geometric properties of the problems such as the null-space property [17], which is also an important ingredient for the study of some more efficient non-convex algorithms [21, 42].

Approaches via descent cone analysis [13], in contrast, provide much more geometric insight. The underlying idea of such approaches is to study the minimum conic singular value defined by

[TABLE]

for $\mathcal{K}$ the descent cone of the underlying atomic norm – the nuclear norm in case of low-rank matrix recovery. For a more detailed review of this approach including a precise definition of the descent cone we refer to Section 2.3 below. Through the study of the minimum conic singular value many superior results were obtained for low-rank recovery problems, most importantly in the context of phase retrieval [41, 40]. Furthermore, minimum conic singular values can also help to understand certain nonlinear measurement models [52].

For all these reasons, it would be desirable to apply this approach also for matrix completion and blind deconvolution. A challenge that one faces, however, is that for both problems one cannot hope to recover all low-rank matrices; rather, only matrices that satisfy certain coherence constraints are admissible (cf. the discussion in [60, Section 5.4]). In this article we address this challenge, providing the first geometric analysis of these problems. We find that the dimensional factors appearing in the error bounds are the true scaling of the minimum conic singular value and hence intrinsically relate to the underlying geometry. Nevertheless for blind deconvolution, near-optimal recovery is possible, if the noise level is not too small.

1.1 Organization of the paper and our contribution

In Section 2 we will review blind deconvolution, matrix completion, as well as some techniques related to descent cone analysis. In Section 3 we will present the main results of this paper. Theorems 3.1 and 3.5 establish that for both blind deconvolution and matrix completion, nuclear norm minimization is intrinsically ill-conditioned. In contrast, Theorem 3.7 provides a near-optimal error bound for blind deconvolution when the noise level is not too small, implying that the conditioning problems only take effect for very small noise levels. The upper bounds for the minimum conic singular value which are the main ingredients of Theorems 3.1 and 3.5 are derived in Section 4. In Section 5 we prove the stability results for blind deconvolution.

We believe that not only our results, but also the proof techniques and geometric insights in this manuscript will be of general interest and help to obtain further understanding of low-rank matrix recovery models, in particular under coherence constraints. We discuss interesting directions for future research in Section 6.

2 Background and related work

2.1 Blind deconvolution

Blind deconvolution problems arise in a number of different areas in science and engineering such as astronomy, imaging, and communications. The goal is to recover both an unknown signal and an unknown kernel from their convolution. In this paper we work with the circular convolution, which is defined by

[TABLE]

where the index difference $k-j$ is considered modulo $L$. Without further assumptions on $w$ and $x$ this bilinear map is far from injective. Consequently, it is crucial to impose structural constraints on both $w$ and $x$. Arguably, the simplest such model is given by linear constraints, that is, both $w$ and $x$ are constrained to known subspaces. Such a model is reasonable in many applications. In wireless communication, for example, it makes sense to assume that the channel behaviour is dominated by the most direct paths and for the signal $x$ a subspace model can be enforced by embedding the message via a suitable coding map into a higher-dimensional space before transmission.

The first rigorous recovery guarantees for such a model were derived by Ahmed, Recht, and Romberg [1]. More precisely, they assume that $w=Bh$, where $B\in\mathbb{C}^{L\times K}$ is a fixed, deterministic matrix such that $B^{*}B=Id_{K}$ (i.e., $B$ is an isometry) and they model $x=C\overline{m_{0}}$, where $\overline{m}_{0}$ denotes the complex-conjugate of $m_{0}$. Here, the matrix $C\in\mathbb{C}^{L\times K}$ is a random matrix, whose entries are independent and identically distributed with circular symmetric normal distribution $\mathcal{CN}\left(0,\frac{1}{\sqrt{L}}\right)$. In this paper we also adopt this model.

Using the well-known fact that the Fourier transform diagonalizes the circular convolution one can rewrite

[TABLE]

where $F\in\mathbb{C}^{L\times L}$ denotes the normalized, unitary discrete Fourier matrix, and

[TABLE]

Denoting by $b_{\ell}$ the $\ell$th row of the matrix $\overline{FB}$, and by $c_{\ell}$ the $\ell$th row of the matrix $\sqrt{L}FC$, one observes that

[TABLE]

Furthermore, because of the rotation invariance of the circular symmetric normal distribution all the entries of the vectors $\left\{c_{\ell}\right\}_{\ell=1}^{L}$ are (jointly) independent and identically distributed with distribution $\mathcal{CN}\left(0,1\right)$. Noting that the expression $\langle h_{0}m^{*}_{0},b_{\ell}c^{*}_{\ell}\rangle_{F}$ is linear in $h_{0}m^{*}_{0}$, Ahmed, Recht, and Romberg [1] defined the operator $\mathcal{A}:\mathbb{C}^{K\times N}\rightarrow\mathbb{C}^{L}$ by

[TABLE]

obtaining the measurement model

[TABLE]

where $e\in\mathbb{C}^{L}$ is additive noise and $X_{0}=h_{0}m^{*}_{0}$. The goal is then to determine $h_{0}$ and $m_{0}$ from $y\in\mathbb{C}^{L}$ up to the inherent scaling ambiguity, or, equivalently, to find the rank-one matrix $X_{0}=h_{0}m^{*}_{0}$.

For $e=0$, among all solutions of the equation $y=\mathcal{A}\left(X_{0}\right)$, the matrix $X_{0}$ is the one with the smallest rank. For this reason, Ahmed, Recht, and Romberg [1] suggested minimizing a natural proxy for the rank, the nuclear norm $\|\cdot\|_{\ast}$, defined as the sum of the singular values of a matrix.

[TABLE]

Here $\tau>0$ is an a priori bound for the noise level, that is, we assume that $\|e\|\leq\tau$. For this semidefinite program, they establish the following recovery guarantee.

Theorem 2.1** ([1]).**

Consider measurements of the form $y=\mathcal{A}\left(h_{0}m^{*}_{0}\right)+e$ for $h_{0}\in\mathbb{C}^{K}$, $m_{0}\in\mathbb{C}^{N}$, $e\in\mathbb{C}^{L}$, and $\mathcal{A}$ as defined in (2). Assume that $\|e\|\leq\tau$ and

[TABLE]

Then with probability exceeding $1-\mathcal{O}\left(L^{-1}\right)$ every minimizer $\hat{X}$ of the SDP (3) satisfies

[TABLE]

Here $\mu^{2}_{\max}$ and $\mu^{2}_{h_{0}}$ are coherence parameters, which are defined via

[TABLE]

and

[TABLE]

The third coherence factor $\tilde{\mu}_{h_{0}}$ is a technical term corresponding to a partition that is constructed as a part of the proof of Theorem 2.1, which is based on the Golfing Scheme [25].

To put the impact of the coherence factors into perspective, observe that if all vectors $b_{\ell}$ have the same $\ell_{2}$-norm, one obtains that $\mu_{\max}=1$; this will be the case, for example, when $B$ is a low-frequency Fourier matrix, as it appears for applications in wireless communication. The second coherence factor always satisfies $1\leq\mu^{2}_{h_{0}}\leq K\mu^{2}_{\max}$. If $\mu_{h_{0}}$ is smaller, this indicates that the mass $\|h_{0}\|^{2}=\sum_{\ell=1}^{L}|\langle b_{\ell},h_{0}\rangle|^{2}$ is distributed fairly evenly among $|\langle b_{\ell},h_{0}\rangle|$. For example, if $\mu_{h_{0}}=1$, then $|\langle b_{\ell},h_{0}\rangle|=\frac{1}{\sqrt{L}}\|h_{0}\|$ for all $\ell\in\left[L\right]$. Numerical simulations in [1] confirm that many $h_{0}$ corresponding to large $\mu_{h_{0}}$ show worse performance, indicating that this factor may be necessary.

The last coherence factor $\tilde{\mu}_{h_{0}}$, in contrast, will no longer appear in our result below, which is why we refrain from detailed discussion. We refer the interested reader to [46, Remark 2.1] and [30, Section 2.3] for details.

For generic $h_{0}$ the parameters $\mu_{h_{0}}$ and $\tilde{\mu}_{h_{0}}$ are reasonably small. For example, if $h_{0}$ is chosen from the uniform distribution on the sphere, one can show that with high probability $\mu_{h_{0}}=\mathcal{O}\left(\sqrt{\log L}\right)$.

For the noiseless case, i.e., $\tau=0$, Theorem 2.1 yields exact recovery, and the required sample complexity $L/\log^{3}L\gtrsim K+N$ is optimal up to logarithmic factors, as the number of degrees of freedom is $K+N-1$ (see [32] for an exact identifiability analysis based on algebraic geometry.) However, if there is noise, the bound for the reconstruction error scales with $\sqrt{K+N}$, in contrast to other measurement scenarios such as low-rank matrix recovery from Gaussian measurements (see, e.g., [13]).

Let us comment on some related work. The foundational paper [1] has triggered a number of follow-up works on the problem of randomized blind deconvolution. A first line of works extended the result to recovering signals from their superposition $\sum_{i=1}^{r}w_{i}\ast x_{i}$, a problem often referred to as blind demixing [46, 30]. Another line of works investigated non-convex (gradient-descent based) algorithms [45, 48, 28], which have the advantage that they are computationally less expensive, as they operate in the natural parameter space. It has been shown that they require a near-optimal number of measurements for recovery. For such an algorithm, [45] derived near-optimal noise-bounds for a Gaussian noise model. However, as in this paper, we focus on the scenario of adversarial noise (instead of random noise) the resulting guarantees are not comparable to ours below.

2.2 Matrix completion

The matrix completion problem of reconstructing a low-rank matrix $X_{0}\in\mathbb{R}^{n_{1}\times n_{2}}$ (we assume that w.l.o.g. $n_{1}\geq n_{2}$) from only a part of its entries arises in many different applications such as in collaborative filtering [56] and multiclass learning [3]. For this reason one could observe a flurry of work on this problem in the last decade, and we will only be able to give a very selective overview of this topic. The precise sampling model that we consider is that $m$ entries of $X_{0}$ are sampled uniformly at random with replacement. Denoting by $e_{i}$ the standard coordinate vectors in $\mathbb{R}^{n_{1}}$ and $\mathbb{R}^{n_{2}}$, respectively, the corresponding measurement operator $\mathcal{A}:\mathbb{R}^{n_{1}\times n_{2}}\rightarrow\mathbb{R}^{m}$ can be written as

[TABLE]

where $\left(a_{i},b_{i}\right)\in\left[n_{1}\right]\times\left[n_{2}\right]$ is chosen uniformly at random for each $i\in\left[m\right]$ (and independently from all other measurements). The scaling factor $\sqrt{\frac{n_{1}n_{2}}{m}}$ in the definition of the measurement operator $\mathcal{A}$ is chosen to ensure that $\mathbb{E}\left[\|\mathcal{A}\left(X\right)\|^{2}\right]=\|X\|_{F}^{2}$. (Some other papers on matrix completion choose a different scaling. We have chosen this normalization because in this way the results for the matrix completion problem can be better compared to those for the blind deconvolution scenario.) Alternative sampling models analyzed in other works include sampling a subset $\Omega$ uniformly from $\left[n_{1}\right]\times\left[n_{2}\right]$ (i.e., without replacement, see, e.g., [10]), or sampling using random selectors.

Again we aim to recover $X_{0}$ from noisy observations $y=\mathcal{A}\left(X_{0}\right)+e$, with a noise vector $e\in\mathbb{R}^{m}$ that satisfies $\|e\|\leq\tau$ via the SDP

[TABLE]

For matrix completion, this approach has first been studied in [8].

It is well known that similarly to the blind deconvolution problem, some incoherence assumptions are necessary to allow for successful recovery. Indeed, suppose that $X_{0}=e_{1}e^{*}_{1}$. Then, if $m\ll n_{1}n_{2}$ with high probability it holds that $\mathcal{A}\left(X_{0}\right)=0$ and one cannot hope to recover $X_{0}$. To avoid such special cases, one needs to ensure that the mass of the Frobenius norm of $X_{0}$ is spread out over all entries rather evenly. If $U\Sigma V^{T}$ is the singular value decomposition of the rank-$r$ matrix $X_{0}$ (with $\Sigma\in\mathbb{R}^{r\times r}$), then this property is captured by the following coherence parameters [25]

[TABLE]

For these coherence parameters, a series of works [8, 10, 25, 54, 14, 19] lead to the following recovery guarantee for the noiseless scenario.

Theorem 2.2** ([19]).**

Consider measurements of the form $y=\mathcal{A}\left(X_{0}\right)$, where $X_{0}\in\mathbb{R}^{n_{1}\times n_{2}}$ is a rank-$r$ matrix with singular value decomposition $X_{0}=U\Sigma V^{T}$ and $\mathcal{A}$ is given by (5). Assume that

[TABLE]

*Then with probability at least $1-\mathcal{O}\left(n^{-1}_{1}\right)$ the matrix $X_{0}$ is the unique minimizer of the SDP (6) with $\tau=0$. *

As for blind deconvolution, this result has been shown using an approximate dual certificate. In [7] this result has been generalized to the case of adversarial noise, showing that with high probability the minimizer $\hat{X}$ of (6) satisfies

[TABLE]

whenever $m\gtrsim n_{1}\text{polylog}\ n_{1}$. As in the blind deconvolution framework, this error bound differs from the case of full Gaussian measurements as discussed, for example, in [13], and also from oracle estimates [6, Section III.B] by a dimensional scaling factor, which will be addressed in this paper.

Also random noise models for matrix completion have been studied in a number of works. In particular, we would like to mention [36, 51], which derive near-optimal rates (both in sample size and estimation error) for matrix completion under subexponential noise with a slightly different nuclear-norm penalized estimator than the one we consider as long as the noise-level is not too small. Similar bounds have also been obtained in [35] using an estimator, which is closer to the one in this work.

Apart from convex methods also many nonconvex algorithms have been proposed and analysed, for example a number of variants of gradient descent (see, e.g., [33, 29, 27, 59, 23, 21, 42, 48]). Arguably the strongest result for matrix completion under adversarial noise has been shown in [33, 34]. These works propose a non-convex algorithm based on Riemannian optimization and show that if the number of measurements is larger than $r^{2}n_{1}\text{polylog}\left(n_{1}\right)$ the true matrix can be reconstructed up to an estimation error superior to the one in [7]. Namely for $\kappa$ denoting the condition number of the matrix $X_{0}$ they show that the output $\hat{X}$ of their algorithm satisfies (in our notation)

[TABLE]

provided the noise level is below a certain, small threshold that scales with the smallest singular value of $X_{0}$. For error vectors $e$ that are spread out evenly and matrices that are well conditioned, one has that $\sqrt{m}\|e\|_{\infty}\approx\|e\|_{2}$, so this bound is superior to (7) in the sense that the scaling factors that appear only scale with the rank $r$ and not the dimension. It should be noted though that in contrast to nuclear norm minimization the underlying algorithm requires precise knowledge of the true rank of the matrix to be recovered.

Just before completion of this manuscript, Chen et al. [16] bridged convex and nonconvex approaches, using nonconvex methods to analyze a convex recovery scheme. Their results provide near optimal recovery guarantees for the matrix completion problem via nuclear norm minimization under a subgaussian random noise model for a much larger range of admissible noise levels than the aforementioned works. More precisely, the proof is based on the observation that in their scenario the minimizer of the convex problem is very close to an approximate critical point of a non-convex gradient based method. This allows them to transfer existing stability results [48] for non-convex optimization to the convex problem. However, the required sample complexity scales suboptimally in the rank $r$ of the matrix and similarly to (8), the error bound depends on the condition number $\kappa$.

2.3 Descent cone analysis

In recent years a number of works have studied low-rank matrix recovery and compressed sensing via a descent cone analysis. This approach has been pioneered for $\ell_{1}$-norm minimization in [58] and for more general (atomic) norms in [13]. Here the descent cone of a norm at a point $X_{0}\in\mathbb{C}^{K\times N}$ is the set of all possible directions $Z\in\mathbb{C}^{K\times N}$ such that the norm does not increase. For the nuclear norm, this leads to the following definition.

Definition 2.3**.**

For any matrix $X_{0}\in\mathbb{C}^{K\times N}$ define its descent cone $\mathcal{K}_{\ast}\left(X_{0}\right)$ by

[TABLE]

To understand its relevance for recovery guarantees assume for a moment that we are in the noiseless scenario, i.e., $\tau=0$ and $e=0$. Then the matrix $X_{0}\in\mathbb{C}^{K\times N}$ is the unique minimizer the semidefinite program (3), if and only if the null space of $\mathcal{A}$ does not intersect the descent cone $\mathcal{K}_{\ast}\left(X_{0}\right)$. In the case of noise, the constraint $\|y-\mathcal{A}\left(X_{0}\right)\|\leq\tau$ in the SDPs (3) and (6) defines a region around $X_{0}+\text{ker}\mathcal{A}$, i.e., the affine subspace consistent with the observed measurements in the noiseless scenario. The intersection of this region with the set of all signals that have a smaller nuclear norm than the ground truth $X_{0}$ is the set of feasible solutions that are preferred to $X_{0}$. The following quantity for a matrix $X_{0}$, which is often referred to as minimum conic singular value, quantifies the size of this intersection

[TABLE]

If $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(X_{0}\right)\right)$ becomes larger, this intersection becomes smaller, which translates into stronger recovery guarantees. The following theorem confirms this intuition.

Theorem 2.4**.**

[13, Proposition 2.2]** Let $\mathcal{A}:\mathbb{C}^{n_{1}\times n_{2}}\rightarrow\mathbb{C}^{m}$ be a linear operator and assume that $y=\mathcal{A}\left(X_{0}\right)+e$ with $\|e\|\leq\tau$. Then any minimizer $\hat{X}$ of the SDP (3) satisfies

[TABLE]

When measurement matrices of the operator $\mathcal{A}$ are full Gaussian matrices (in contrast to rank-$1$ measurements as in this paper) and $\mathcal{A}$ is normalized such that $\mathbb{E}\left[\mathcal{A}^{*}\mathcal{A}\right]=\text{Id}$, for an arbitrary low-rank matrix $X_{0}$ one has with high probability that $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(X_{0}\right)\right)\asymp 1$. Consequently, Theorem 2.4 yields an optimal estimation error even for adversarial noise. As we will show this is no longer the case for blind deconvolution and matrix completion.

The geometric analysis of linear inverse problems via the descent cone and the minimum conic singular value has lead to many new results and insights in compressed sensing and low-rank matrix recovery. For convex programs the phase transition of the success rate could be precisely predicted [2]. As the proofs are specific to full Gaussian measuements, they do not apply for a number of important structured and heavy-tailed measurement scenarios. Stronger results [43, 20, 40, 31, 41] were subsequently obtained using Mendelson’s small ball method [37, 50], a powerful tool for bounding a nonnegative empirical process from below, now often refereed to as Mendelson’s small ball method.

2.4 Notation

For $n\in\mathbb{N}$ we will write $\left[n\right]$ to denote the set $\left\{1;\ldots;n\right\}$. For any set $A$ we will denote its cardinality by $|A|$. For a complex number $z$ we will denote its real part by $\text{Re}\left(z\right)$ and its imaginary part by $\text{Im}\left(z\right)$. By $\log\left(\cdot\right)$ we will denote the logarithm to the base $e$. By $\mathbb{E}X$ we will denote the expectation of a random variable $X$ and by $\mathbb{P}\left(A\right)$ we denote the probability of an event $A$. If $v\in\mathbb{C}^{n}$ we will denote its $\ell_{2}$-norm by $\|v\|$ and its Hermitian transpose by $v^{*}$. For $u,v\in\mathbb{C}^{n}$ the (Euclidean) inner product is defined by $\langle u,v\rangle:=u^{*}v$. Furthermore, for $Z\in\mathbb{C}^{n_{1}\times n_{2}}$ its spectral norm is given by $\|Z\|$, i.e., the dual norm of the nuclear norm $\|Z\|_{\ast}$. Moreover, the Frobenius norm of $Z$ is defined by $\|Z\|_{F}$ with corresponding inner product $\langle Z,W\rangle_{F}:=\text{Tr}\,\left(Z^{*}W\right)$, where $W\in\mathbb{C}^{n_{1}\times n_{2}}$. When we study matrix completion, we will work with matrices $Z\in\mathbb{R}^{n_{1}\times n_{2}}$ and the previous quantities will be defined analogously. Moreover, in that scenario we will use the notation $\|Z\|_{\ell_{\infty}}:=\underset{\left(i,j\right)\in\left[n_{1}\right]\times\left[n_{2}\right]}{\max}|Z_{i,j}|$, where $\left\{Z_{i,j}\right\}^{n_{1},n_{2}}_{i,j=1}$.

3 Our results

3.1 Instability of low-rank matrix recovery

3.1.1 Blind deconvolution

Our first main result states that randomized blind deconvolution can be unstable under adversarial noise.

Theorem 3.1**.**

Let $K,N\in\mathbb{N}\setminus\left\{1\right\}$. Assume that

[TABLE]

*Then there exists a matrix $B\in\mathbb{C}^{L\times K}$ satisfying $B^{*}B=\text{Id}_{K}$ and with $FB$ having rows of equal norm, i.e., $\mu^{2}_{\max}=1$, such that for all $h_{0}\in\mathbb{C}^{K}\setminus\left\{0\right\}$ and $m_{0}\in\mathbb{C}^{N}\setminus\left\{0\right\}$ the following holds:

With probability at least $1-\mathcal{O}\left(\exp\left(-\frac{K}{C_{2}\mu_{h_{0}}^{2}}\right)\right)$, where $\mu^{2}_{h_{0}}=\frac{L}{\|h_{0}\|^{2}}\underset{\ell\in\left[L\right]}{\max}\ |\langle b_{\ell},h_{0}\rangle|^{2}$, there is $\tau_{0}>0$ such that for all $\tau\leq\tau_{0}$ there exists an adversarial noise vector $e\in\mathbb{C}^{L}$ with $\|e\|\leq\tau$ that admits an alternative solution $\tilde{X}$ with the following properties.*

• •

$\tilde{X}$* is feasible, i.e., $\|\mathcal{A}\left(\tilde{X}\right)-y\|=\tau$ for $y=\mathcal{A}\left(h_{0}m^{*}_{0}\right)+e$ the noisy measurement vector*

• •

$\tilde{X}$* is preferred to $X_{0}=h_{0}m^{*}_{0}$ by the SDP (3), i.e., $\|\tilde{X}\|_{\ast}\leq\|X_{0}\|_{\ast}$, but*

• •

$\tilde{X}$* is far from the true solution in Frobenius norm, i.e.,*

[TABLE]

The constants $C_{1}$, $C_{2}$, and $C_{3}$ are universal.

Remark 3.2**.**

*The matrix $B$ in the above result exactly fits into the framework of Theorem 2.1. Indeed, one can check that for many interesting cases (including the case that $K$ divides $L$) it holds that $\tilde{\mu}_{h_{0}}\lesssim\mu_{h_{0}}$. That is, the assumptions of Theorem 2.1 cannot be enough to deduce stability.

We do not expect, however, that this kind of instability is observed for arbitrary isometric embeddings $B\in\mathbb{C}^{L\times K}$. For example, let $B$ be a random embedding, which is chosen from the uniform distribution over the Stiefel manifold $\mathbb{V}^{L}_{K}$, i.e., the manifold consisting of all matrices $\tilde{B}\in\mathbb{C}^{L\times K}$ such that $\tilde{B}^{*}\tilde{B}=\text{Id}_{K}\in\mathbb{C}^{K\times K}$. In this case, we expect that a similar proof as in [53, 40] applies and that the multiplicative dimensional factor does not appear in the error bound with high probability if one randomizes over $B$ and $C$ simultaneously. In particular, this implies the existence of an isometric embedding $B$ such that a result analogous to Theorem 3.1 cannot hold.

An interesting open problem is whether the statement of Theorem 3.1 still holds if $FB$ is a low-frequency discrete Fourier matrix, which is a common assumption in blind deconvolution. The corresponding $b_{\ell}$’s should lead to better conditioning as in our counterexample, but worse than in the case of a random $B$, as a number of $b_{\ell}$’s exhibit substantial correlation, but many are uncorrelated. In that sense, this scenario is in between the scenario of arbitrary $B$’s sketched above and the adversarial scenario in Theorem 3.1.*

To put our results in perspective note that for $L\asymp\left(K+N\right)\text{polylog}\left(K+N\right)$, which is the minimal number of measurements required for noiseless recovery, it holds that $\sqrt{\frac{KN}{L}}\asymp\sqrt{\frac{\min\left\{K,N\right\}}{\text{polylog}\left(K+N\right)}}$. Up to logarithmic factors, this coincides with the rate predicted by (4), whenever $K\asymp N$.

Theorem 3.1 is a direct consequence of the following proposition, which we think is interesting in its own right.

Proposition 3.3**.**

Let $K,N\in\mathbb{N}\setminus\left\{1\right\}$. Assume that

[TABLE]

*Then there exists $B\in\mathbb{C}^{L\times K}$ satisfying $B^{*}B=\text{Id}_{K}$ and $\mu^{2}_{\max}=1$, whose corresponding measurement operator $\mathcal{A}$ satisfies the following.

Let $h_{0}\in\mathbb{C}^{K}\setminus\left\{0\right\}$, $m_{0}\in\mathbb{C}^{N}\setminus\left\{0\right\}$ and define $\mu_{h_{0}}$ as in Theorem 3.1. Then with probability at least $1-\mathcal{O}\left(\exp\left(-C_{2}K/\mu_{h_{0}}^{2}\right)\right)$ it holds that*

[TABLE]

Here $C_{1}$, $C_{2}$, and $C_{3}$ are absolute constants.

The proof of Proposition 3.3 will be provided in Section 4. Note that by definition of the minimum conic singular value $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\right)$ Proposition 3.3 is equivalent to the statement that with high probability there is $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\setminus\left\{0\right\}$ such that

[TABLE]

Our construction of such $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$ relies on the observation that with high probability there is a rank-one matrix $W\in\mathbb{C}^{K\times N}$ in the null-space of $\mathcal{A}$ which is relatively close to the descent cone (with respect to the $\|\cdot\|_{F}$-distance). Perturbing $W$ by $-\beta h_{0}m^{*}_{0}$ for a suitable $\beta$ one can then obtain a matrix $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$, which fulfills (11).

The existence of such a matrix $W\in\text{ker}\mathcal{A}$ also reveals a fact about the geometry of the problem, which we find somewhat surprising: while the null space of $\mathcal{A}$ does not intersect the descent cone (otherwise exact recovery would not be possible), the angle between those objects is very small. This is very different from the behavior for measurement matrices $\mathcal{A}$ with i.i.d. Gaussian entries (instead of $b_{\ell}c^{*}_{\ell}$).

Remark 3.4**.**

While $\tilde{X}$ is preferred to the true solution by the SDP (3) $\tilde{X}$ is typically not a minimizer of (3). To see this, assume that without noise exact recovery is possible, which is the case with high probability by Theorem 2.1. Then consider $\tilde{X}=X_{0}+tZ$ for $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$ of the form $Z=W-\beta h_{0}m^{*}_{0}$ with $W\in\text{ker}\mathcal{A}$ and $\beta>0$ such that $\frac{\|\mathcal{A}\left(Z\right)\|}{\|Z\|_{F}}\lesssim\sqrt{\frac{L}{KN}}$, as in the proof of Proposition 3.3. As $W\notin\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$ (otherwise exact recovery would not be possible) it follows that for $t>0$

[TABLE]

where the last line is due to $\mathcal{K}_{\ast}\left(X_{0}\right)=\mathcal{K}_{\ast}\left(\left(1-t\beta\right)X_{0}\right)$.

*On the other hand, we also have that $\mathcal{A}\left(\hat{X}\right)=\mathcal{A}\left(\left(1-t\beta\right)X_{0}\right)$ due to $\mathcal{A}\left(W\right)=0$ and, hence, $\left(1-t\beta\right)X_{0}$ is admissible whenever $\tilde{X}$ is admissible. Consequently, the SDP (3) will always prefer $\left(1-\beta t\right)h_{0}m^{*}_{0}$ to $\tilde{X}$ and $\tilde{X}$ will never be a minimizer. It remains an open problem what one can say about the minimizer $\hat{X}$ of (3), see also Section 6.

Even if the minimizer of (3) $\hat{X}$ is closer to the ground truth (in $\|\cdot\|_{F}$-distance) than $\tilde{X}$, however, the nuclear norms of $X$ and $\tilde{X}$ will be very close, which can easily lead to numerical instabilities.*

3.1.2 Matrix completion

Our second main result states that for arbitrary incoherent low-rank matrices, matrix completion is unstable with high probability. Note that in contrast to Theorem 3.1 which is based on a specific choice of parameters the following result holds for an arbitrary incoherent matrix $X_{0}$.

Theorem 3.5**.**

Let $n_{1}\geq n_{2}$ and let $\mathcal{A}:\mathbb{R}^{n_{1}\times n_{2}}\rightarrow\mathbb{R}^{m}$ be defined as in (5). Assume that $X_{0}\in\mathbb{R}^{n_{1}\times n_{2}}\setminus\left\{0\right\}$ is a rank $r$ matrix with singular value decomposition $X_{0}=U\Sigma V^{*}$. Moreover, assume that

[TABLE]

Then with probability at least $1-\mathcal{O}\left(\exp\left(-\frac{m}{C_{2}r\mu^{2}\left(U\right)\mu^{2}\left(V\right)}\right)\right)$ there is $\tau_{0}>0$ such that for all $\tau\leq\tau_{0}$ there exists an adversarial noise vector $e\in\mathbb{R}^{m}$ with $\|e\|\leq\tau$ that admits an alternative solution $\tilde{X}\in\mathbb{R}^{n_{1}\times n_{2}}$ with the following properties.

• •

$\tilde{X}$* is feasible, i.e., \Big{\|}\mathcal{A}\left(\tilde{X}\right)-y\Big{\|}=\tau for $y=\mathcal{A}\left(X_{0}\right)+e$ the noisy measurement vector*

• •

$\tilde{X}$* is preferred to $X_{0}$ by the SDP (6), i.e., $\|\tilde{X}\|_{\ast}\leq\|X_{0}\|_{\ast}$ , but*

• •

$\tilde{X}$* is far from the true solution in Frobenius norm, i.e.,*

[TABLE]

Here the constants $C_{1}$, $C_{2}$, and $C_{3}$ are universal.

Again, to put our results in perspective note that for $m\asymp n_{1}\text{polylog}\left(n_{1}\right)$, which is the minimal number of measurements required for noiseless recovery, it holds that $\sqrt{\frac{rn_{1}n_{2}}{m}}\asymp\sqrt{\frac{n_{2}}{\text{polylog}\left(n_{1}\right)}}$. Up to logarithmic factors, this coincides with the rate predicted by (7).

Theorem 3.5 is a direct consequence of the following proposition, which, in our opinion, is of independent interest, as it provides a negative answer to a question by Tropp [60, Section 5.4].

Proposition 3.6**.**

Let $X_{0}\in\mathbb{R}^{n_{1}\times n_{2}}\setminus\left\{0\right\}$ be a rank-$r$ matrix with corresponding singular value decomposition $X_{0}=U\Sigma V^{*}$. Moreover, assume that

[TABLE]

Then with probability at least $1-\mathcal{O}\left(\exp\left(-\frac{m}{C_{2}r\mu^{2}\left(U\right)\mu^{2}\left(V\right)}\right)\right)$ it holds that

[TABLE]

The constants $C_{1}$, $C_{2}$, and $C_{3}$ are universal.

Proposition 3.6 corresponds to Proposition 3.3 for blind deconvolution and will be proved analogously. We will again show that with high probability there is $W\in\mathbb{R}^{n_{1}\times n_{2}}$ such that $\mathcal{A}\left(W\right)=0$ and $W$ is relatively close to the descent cone of $X_{0}$ in $\|\cdot\|_{F}$-distance. Setting $Z:=W-\beta UV^{*}$ for a suitable $\beta>0$ yields an element of $\mathcal{K}_{\ast}\left(X_{0}\right)$ with

[TABLE]

3.2 Stable recovery

A geometric interpretation of Theorems 3.1 and 3.5 is that the nuclear norm ball is near-tangential to both the kernels of matrix completion and randomized blind deconvolution. Given that tangent spaces only provide local approximation, these results leave open, what happens in some distance, i.e., for larger noise levels – this will depend on the curvature of the nuclear norm ball.

Our third main result concerns exactly this problem for the randomized blind deconvolution setup. As it turns out, the descent directions $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$ with $\|\mathcal{A}\left(Z\right)\|/\|Z\|_{F}$ very small correspond to directions of significant curvature. That is, only a very short segment in this direction will have smaller nuclear norm than $h_{0}m^{*}_{0}$, and the corresponding alternative solutions all correspond to very small $e$. For noise levels $\tau$ large enough, in contrast, these directions can be excluded and one can obtain near-optimal error bounds. In order to precisely formulate this observation, we denote the set of $\mu$-incoherent vectors $h\in\mathbb{C}^{K}$ with respect to $B\in\mathbb{C}^{L\times K}$ for $\mu\geq 1$ by

[TABLE]

With this notation, our result reads as follows.

Theorem 3.7**.**

Let $\alpha>0$, $\mu\geq 1$, and $B\in\mathbb{C}^{L\times K}$ such that $B^{*}B=\text{Id}$. Assume that

[TABLE]

*Then with probability at least $1-\mathcal{O}\left(\exp\left(-\frac{L\alpha^{4/3}}{C_{2}\log^{4/3}\left(eL\right)\mu^{4/3}}\right)\right)$ the following statement holds for all $h_{0}\in\mathcal{H}_{\mu}\setminus\left\{0\right\}$, all $m_{0}\in\mathbb{C}^{N}\setminus\left\{0\right\}$, all $\tau>0$, and all $e\in\mathbb{C}^{L}$ with $\|e\|\leq\tau$ :

Any minimizer $\hat{X}$ of (3) satisfies*

[TABLE]

Here $C_{1}$, $C_{2}$, and $C_{3}$ are absolute constants.

In words, this theorem establishes linear scaling in the noise level $\tau$ with only a logarithmic dimensional factor for $\tau\geq\alpha\|h_{0}m^{*}_{0}\|_{F}$, in contrast to the polynomial factor required for small noise levels as a consequence of Theorem 3.1. Here the value of $\alpha$ can be chosen arbitrarily small, at the expense of an increased number of measurements. For example when one is interested in noise levels $\tau=\epsilon\mu^{-2}\log^{-2}L$ for some $\epsilon>\epsilon_{0}$ (this is the largest order to expect meaningful error bounds despite the additional logarithmic factors) one should choose $\alpha\asymp\epsilon_{0}\mu^{-2}\log^{-2}L$, and near-linear error bounds will be guaranteed for a sample complexity of

[TABLE]

Remark 3.8**.**

A similar approach to the proof of Theorem 3.7 also yields a corresponding result for rank-one matrix completion. Arguably, however, matrix completion is mainly of interest for ground truth matrices of rank higher than one, so we decided to omit the proof details.

4 Upper bounds for the minimum conic singular values

4.1 Characterization of the descent cone of the nuclear norm

The goal of this section is to prove Proposition 3.3 and Proposition 3.6, from which we will then be able to deduce Theorem 3.1 and Theorem 3.5. For that we first discuss a characterization of the descent cone $\mathcal{K}_{\ast}\left(X\right)$. In order to state this characterization, Lemma 4.1, we need to introduce some additional notation. Let $X\in\mathbb{C}^{n_{1}\times n_{2}}$ be a matrix of rank $r$. We will denote its corresponding singular value decomposition by $X=U\Sigma V^{*}$, where $\Sigma\in\mathbb{R}^{r\times r}$ is a diagonal matrix with nonnegative entries and $U\in\mathbb{C}^{n_{1}\times r}$ and $V\in\mathbb{C}^{n_{2}\times r}$ are unitary matrices, i.e., $U^{*}U=V^{*}V=\text{Id}_{r}$. This allows us to define the tangent space of the manifold of rank-$r$ matrices at the point $X$ by

[TABLE]

By $\mathcal{P}_{T_{X}}$ we will denote the orthogonal projection onto $T_{X}$, by $\mathcal{P}_{T^{\perp}_{X}}=\text{Id}-\mathcal{P}_{T_{X}}$ the projection onto its orthogonal complement.

Lemma 4.1**.**

Let $X\in\mathbb{C}^{n_{1}\times n_{2}}\backslash\left\{0\right\}$ be a matrix of rank $r$ with corresponding singular value decomposition $X=U\Sigma V^{*}$. Then

[TABLE]

where $\overline{\mathcal{K}_{\ast}\left(X\right)}$ denotes the topological closure of $\mathcal{K}_{\ast}\left(X\right)$.

Remark 4.2**.**

Lemma 4.1 is similar to well-known results in convex optimization and may be known to the community. As we could not find it in the literature in this form, we decided to include a proof for completeness.

The proof of Lemma 4.1 relies on the duality between the descent cone and the subdifferential of a convex function. In the following we will denote by $\partial\|\cdot\|_{\ast}\left(X\right)$ the subdifferential of the nuclear norm at the point $X\in\mathbb{C}^{n_{1}\times n_{2}}$. We will use that a characterization of $\partial\|\cdot\|_{\ast}$ is well-known [64]. Namely, for all $X\in\mathbb{C}^{n_{1}\times n_{2}}$ with corresponding singular value decomposition $X=U\Sigma V^{*}$ it holds that

[TABLE]

Proof.

Recall that for a set of matrices $\mathcal{V}\subset\mathbb{C}^{n_{1}\times n_{2}}$ its polar cone $\mathcal{V}^{\circ}$ is defined by

[TABLE]

For all $X\in\mathbb{C}^{n_{1}\times n_{2}}\backslash\left\{0\right\}$ we have the following polarity relation between the descent cone and the subdifferential

[TABLE]

For sets and functions defined in $\mathbb{R}^{n}$ with the usual Euclidean inner product, this is [57, Theorem 23.7]. The complex case directly follows, as $\mathbb{C}^{n_{1}\times n_{2}}$ with the inner product $\text{Re}\left(\langle\cdot,\cdot\rangle_{F}\right)$ can be identified with an $2n_{1}n_{2}$-dimensional real-valued vector space with standard Euclidean inner product.

It follows from the bipolar theorem (see, e.g., [5, p. 53]) that

[TABLE]

Hence, in order to complete the proof it is sufficient to show that

[TABLE]

First, suppose that $Z\in\mathbb{C}^{n_{1}\times n_{2}}$ satisfies $-\text{Re}\left(\langle UV^{*},Z\rangle_{F}\right)\geq\|\mathcal{P}_{T^{\perp}_{X}}\left(Z\right)\|_{\ast}$. We have to show that $\text{Re}\left(\langle W,Z\rangle_{F}\right)\leq 0$ for all $W\in\partial\|\cdot\|_{\ast}\left(X\right)$. Indeed,

[TABLE]

In the first inequality we have used that the spectral norm is the dual norm of the nuclear norm. The second inequality follows from $\|\mathcal{P}_{T^{\perp}_{X}}W\|\leq 1$. Hence, we have shown that $Z\in\left(\partial\|\cdot\|_{\ast}\left(X\right)\right)^{\circ}$. Next, let $Z\in\left(\partial\|\cdot\|_{\ast}\left(X\right)\right)^{\circ}$ be arbitrary. Choose $\tilde{W}\in T^{\perp}_{X}$ such that $\text{Re}\left(\langle\tilde{W},Z\rangle_{F}\right)=\|\mathcal{P}_{T^{\perp}_{X}}\left(Z\right)\|_{\ast}$ and $\|\tilde{W}\|\leq 1$. Then by (15) it follows that $UV^{*}+\tilde{W}\in\partial\|\cdot\|_{\ast}\left(X\right)$ and as $Z\in\left(\partial\|\cdot\|_{\ast}\left(X\right)\right)^{\circ}$ we obtain that

[TABLE]

This shows that $-\text{Re}\left(\langle UV^{*},Z\rangle_{F}\right)\geq\|\mathcal{P}_{T^{\perp}_{X}}\left(Z\right)\|_{\ast}$. Hence, we have verified (16), which completes the proof. ∎

4.2 Upper bound for blind deconvolution

The goal of this section is to prove Proposition 3.3. For that we need the following lemma, which is a consequence of the concentration of measure theorem for Lipschitz functions. (For a proof of the real-valued case see, e.g., [63, Lemma 5.3.2]. The complex-case can be shown analogously.)

Lemma 4.3**.**

Let $P:\mathbb{C}^{n}\rightarrow\mathbb{C}^{n}$ be a random projection onto a $k$-dimensional subspace, which is uniformly distributed in the Grassmannian $Gr\left(k,\mathbb{C}^{n}\right)$. Fix $z\in\mathbb{C}^{n}$. Then for all $\varepsilon>0$ with probability at least $1-2e^{-\tilde{c}k\varepsilon^{2}}$ we have that

[TABLE]

where $\tilde{c}>0$ is some absolute constant.

Proof of Proposition 3.3.

A core ingredient of the proof is to find a tight frame $B$ such that each of its frame vectors is orthogonal to all but a near-minimal number of other frame vectors. For such a $B$ we then choose a vector $h$ out of these frame vectors and use it to construct a matrix in the descent cone that is close to the kernel of the measurement map. For that, we exploit that by the choice of $h$, any rank-one matrix $h\tilde{m}^{*}$ will lead to a large part of zero measurements due to the orthogonality. Consequently, there are also many vectors $m_{1}$ and $m_{2}$ such that $hm_{1}^{*}$ and $hm_{2}^{*}$ lead to the same measurements, including some choices such that $hm_{1}^{*}-hm_{2}^{*}$ is not only in the kernel of the measurement map, but also close to the descent cone.

When $K$ divides $L$ a suitable choice for $B$ consists of $\tfrac{L}{K}$ repetitions of a fixed orthogonal basis. When $K$ does not divide $L$, one can still start off in the same way, if one completes the matrix appropriately to obtain a unit norm tight frame without introducing too many pairs of non-orthogonal frame vectors. One way to achieve this is to find a tight frame that consists only of very sparse vectors, as this will also lead to many vanishing inner products. A natural candidate is hence a so-called spectral tetris frame [11], as it has been shown to be maximally sparse [12]. Indeed, our construction uses exactly this frame.

The Spectral Tetris algorithm is based on the observation that the rows of a matrix $G\in\mathbb{R}^{L\times K}$ form a tight frame if and only if the columns of $G$ are orthogonal and of equal norm. It is easy to see that for unit norm tight frames, the column normalization must be $\sqrt{\frac{L}{K}}$. Spectral Tetris starts by greedily filling up the first column of $G$ by choosing the first $\lfloor\tfrac{L}{K}\rfloor$ rows of $G$ to be the first standard basis vector $e_{1}$. The next two frame vectors are chosen of the form $\alpha e_{1}\pm\sqrt{1-\alpha^{2}}e_{2}$, where $\alpha$ is chosen to fulfil the norm constraint of the first column. Next, the second column is greedily filled, then the third, and so on. By construction, the resulting frame will only consist of $1$-sparse vectors and $2$-sparse vectors supported in neighboring entries. Consequently, the sets

[TABLE]

are disjoint.

Moreover, note that it follows from the Spectral Tetris algorithm that

[TABLE]

where in the last inequality we used that by assumption $2K\leq L$.

Without loss of generality we assume that $\|h_{0}\|=\|m_{0}\|=1$ as rescaling does not change the descent cone $\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$. For the proof we will condition on two events. The first event states that

[TABLE]

which by the Bernstein inequality (see, e.g., [63]) is fulfilled with probability at least $1-\exp\left(-cL/\mu^{2}\right)$, where $c>0$ is some numerical constant. To formulate the second event, we define for all natural numbers $1\leq i\leq K/3$ and $c_{j}$ as in Section 2.1

[TABLE]

and denote by $m_{i}^{\parallel}$ the orthogonal projection of $m_{0}$ onto $D_{i}$ and by $m_{i}^{\perp}$ the projection onto $D_{i}^{\perp}$, the orthogonal complement of $D_{i}$. Note that $D_{i}\subset\mathbb{C}^{N}$ is a random subspace of dimension $|\mathcal{B}_{i}|$, distributed uniformly over the Grassmannian $Gr\left(|\mathcal{B}_{i}|,\mathbb{C}^{N}\right)$ due to the rotation invariance of $\mathcal{CN}\left(0,1\right)$. Hence, as $\|m_{0}\|=1$ Lemma 4.3 together with inequality (17) yields that for fixed $i\in\left[\lfloor K/3\rfloor\right]$ with probability at least $1-2\exp\left(-\frac{\hat{c}L}{4K}\right)$ one has

[TABLE]

where $\hat{c}>0$ is some absolute constant. As the matrix $C$ is Gaussian, the different subspaces $D_{i}$’s and hence also the random vectors $\left\{m_{i}^{\parallel}\right\}^{K}_{i=1}$ are independent, so with probability at least

[TABLE]

there exists at least one $k\in\left[\lfloor K/3\rfloor\right]$ such that (19) holds (with $k=i$). Also note that

[TABLE]

which for $C_{1}=\frac{24\log 2}{\hat{c}}$ follows from assumption (9).

To summarize, we have shown that the two events $\mathcal{E}_{1}:=\left\{\|\mathcal{A}\left(h_{0}m^{*}_{0}\right)\|\leq 2\|h_{0}m^{*}_{0}\|_{F}\right\}$ and

[TABLE]

happen with probability at least $1-\mathcal{O}\left(\exp\left(-C_{2}L/\mu^{2}\right)\right)$, where $C_{2}>0$ is an appropriately chosen constant.

Conditional on $\mathcal{E}_{1}$ and $\mathcal{E}_{2}$, we will construct $Z\in\mathbb{C}^{K\times N}$ (depending on the realization of the random matrix $C$) such that $Z\in\overline{\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)}\setminus\left\{0\right\}$ and such that the inequality

[TABLE]

is satisfied. Note that this will complete the proof. Indeed, by definition of the closure and the continuity of $\mathcal{A}$ this implies that there exists $\tilde{Z}\in\mathcal{K}_{\ast}\left(X_{0}\right)$ such that

[TABLE]

which by the definition of $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\right)$ implies that (10) holds with constant $C_{3}=12$.

To construct $Z$ satisfying (20), define

[TABLE]

where $i\in\left[\lfloor K/3\rfloor\right]$ is chosen to satisfy (19). It follows directly from the definition of $W$ that $\|W\|_{F}=1$. We observe that $\mathcal{A}\left(W\right)=0$ as for each $i\in\left[\lfloor K/3\rfloor\right]$ and $\ell\in\left[L\right]$ we either have $\langle e_{i},b_{\ell}\rangle=0$, $\ell\notin\mathcal{B}_{i}$, or $\langle m^{\perp}_{i},c_{\ell}\rangle=0$, if $\ell\in\mathcal{B}_{i}$. Denote by $T=T_{X_{0}}$ the tangent space of the manifold of rank-one matrices at $X_{0}=h_{0}m^{*}_{0}$ as defined in (14) and by $\mathcal{P}_{T}$ and $\mathcal{P}_{T^{\perp}}$ the corresponding orthogonal projections. It follows that

[TABLE]

Thus we have shown that $W$, an element of the null space of $\mathcal{A}$, is close to the tangent space $T$. We will now show that for $\beta=3\sqrt{\frac{L}{KN}}$

[TABLE]

lies in the closure of the descent cone $\overline{\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)}$. For that, we observe that

[TABLE]

and hence Lemma 4.1 entails that $Z\in\overline{\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)}$. Moreover, note that by the triangle inequality and by the assumption $L\leq\frac{1}{36}KN$ it holds that

[TABLE]

These observations together with $\mathcal{A}\left(W\right)=0$ yield that

[TABLE]

This shows (20), as desired.

∎

4.3 Upper bound for matrix completion

In this section, we prove Proposition 3.6. For that we introduce sets $\mathcal{N}_{a}$, $a\in\left[n_{1}\right]$, via

[TABLE]

That is, $\mathcal{N}_{a}$ contains all the indices of the $a$th row of the matrix $X_{0}$, which are observed by the measurements. Furthermore, define by $P_{\mathcal{N}_{a}}\in\mathbb{R}^{n_{2}\times n_{2}}$ the projection onto the coordinates, which are contained in $\mathcal{N}_{a}$, i.e. $P_{\mathcal{N}_{a}}=\sum_{b\in\mathcal{N}_{a}}e_{b}e^{*}_{b}$. By $P_{\mathcal{N}^{\perp}_{a}}=\sum_{b\in\left[n_{2}\right]\setminus\mathcal{N}_{a}}e_{b}e^{*}_{b}$ we denote the coordinate projection onto $\left[n_{2}\right]\setminus\mathcal{N}_{a}$.

We need the following technical lemma.

Lemma 4.4**.**

Let $V\in\mathbb{R}^{n_{2}\times r}$ be an isometry, i.e. $V^{*}V=\text{Id}$. Assume that

[TABLE]

Then with probability at least $1-\mathcal{O}\left(\exp\left(-\frac{m}{C_{2}r\mu^{2}\left(V\right)}\right)\right)$ there exists $a\in\left[n_{1}\right]$ such that

[TABLE]

$C_{1}$* and $C_{2}$ are universal constants.*

Proof.

For each $a\in\left[n_{1}\right]$ we set $\mathcal{I}_{a}:=\left\{i\in\left[m\right]:a_{i}=a\right\}$ and define the event

[TABLE]

We will first derive a lower bound for \mathbb{P}\left(\mathcal{E}_{a}\big{|}\mathcal{I}_{a}\right). For that we note that $\|P_{\mathcal{N}_{a}}V\|^{2}=\|V^{*}P_{\mathcal{N}_{a}}V\|$. Let $v_{1},v_{2},\ldots,v_{n_{2}}$ denote the rows of the matrix $V$. By definition of $\mathcal{N}_{a}$ and $\mathcal{I}_{a}$ it follows that for $X_{i}:=v_{b_{i}}v^{*}_{b_{i}}$

[TABLE]

Here we write $A\preceq B$ for two symmetric matrices $A$ and $B$, if and only if $B-A$ is positive semidefinite. By (24) it is sufficient to bound the probability of the event

[TABLE]

conditionally on $\mathcal{I}_{a}$. To bound $\|\sum_{i\in\mathcal{I}_{a}}X_{i}\|$ we will use the matrix Bernstein inequality (see, e.g., [61, Theorem 6.1.1]) conditionally on $\mathcal{I}_{a}$, which requires as ingredients

[TABLE]

an upper bound for \sigma^{2}\left(\mathcal{I}_{a}\right):=\Big{\|}\mathbb{E}\left[\sum_{i\in\mathcal{I}_{a}}\left(X_{i}-\mathbb{E}\left[X_{i}\right]\right)^{2}\big{|}\mathcal{I}_{a}\right]\Big{\|} and a constant $K>0$ such that $\|X_{i}-\mathbb{E}\left[X_{i}\right]\|\leq K$ almost surely. To bound $\sigma^{2}\left(\mathcal{I}_{a}\right)$ we note that

[TABLE]

where the fourth line is due to the definition of $\mu^{2}\left(V\right)$. This implies that

[TABLE]

To find an appropriate $K>0$ note that almost surely

[TABLE]

where in the fourth line we used that $\underset{i\in\left[n_{2}\right]}{\max}\|v_{i}\|^{2}\geq\frac{r}{n_{2}}=\frac{1}{n_{2}}\sum_{i\in\left[n_{2}\right]}\|v_{i}\|^{2}$. Finally to apply Bernstein inequality we need that the $X_{i}$’s are independent conditionally on $\mathcal{I}_{a}$, which follows from the fact that the $a_{i}$’s and $b_{i}$’s are drawn independently. With these ingredients the matrix Bernstein inequality yields that

[TABLE]

Setting $t=\frac{m}{2n_{1}n_{2}}$ this implies that for fixed $a\in\left[n_{1}\right]$ it holds that

[TABLE]

To complete the proof we restrict our attention to $A:=\left\{a\in\left[n_{1}\right]:\ |\mathcal{I}_{a}|\leq\frac{4m}{3n_{1}}\right\}$ as it follows from (25) that

[TABLE]

and, consequently, for $a\in A$ we obtain that

[TABLE]

As the $\mathcal{E}_{a}$’s only depend on $\left\{b_{i}\right\}_{i\in\mathcal{I}_{a}}$ and are hence independent conditionally on $\mathcal{I}_{\alpha}$, this implies that

[TABLE]

where in the last line we have used assumption (23) with $C_{1}$ large enough. Furthermore, note that

[TABLE]

implies that $|A|\geq\frac{n_{1}}{4}$ almost surely. Hence, it follows that

[TABLE]

This shows that conditional on $\left\{\mathcal{I}_{a}\right\}^{n_{1}}_{a=1}$ we have that almost surely

[TABLE]

Taking expectations yields the claim.

∎

Now we are prepared to give a proof of Proposition 3.6.

Proof of Proposition 3.6.

For the proof we will condition on two events $\mathcal{E}_{1}$ and $\mathcal{E}_{2}$, which we will define in the following. The event $\mathcal{E}_{1}$ is defined by

[TABLE]

Observe that

[TABLE]

where we have set $X_{i}:=\frac{n_{1}n_{2}}{m}|\left(UV^{*}\right)_{a_{i},b_{i}}|^{2}$. Note that one has almost surely that

[TABLE]

where we have applied Cauchy-Schwarz and the definition of $\mu\left(U\right)$ and $\mu\left(V\right)$. Hence, one has

[TABLE]

where we used $\|UV^{*}\|^{2}_{F}=r$ and the previous estimate. Hence, by the Bernstein inequality (see, e.g., [63, Theorem 2.8.4]) we obtain that

[TABLE]

By setting $t=\|UV^{*}\|^{2}_{F}=r$ we observe that $\mathcal{E}_{1}$ holds with probability at least $1-2\exp\left(-\frac{cm}{r\mu^{2}\left(U\right)\mu^{2}\left(V\right)}\right)$.

The second event $\mathcal{E}_{2}$ is defined by

[TABLE]

For $C_{1}$ in assumption (12) chosen large enough Lemma 4.4 then entails that $\mathbb{P}\left(\mathcal{E}_{2}\right)\geq 1-\mathcal{O}\left(\exp\left(-\frac{cm}{r\mu^{2}\left(V\right)}\right)\right)$. Consequently, we can find $a\in\left[n_{1}\right]$ (depending on the random sampling pattern) such that the condition defining $\mathcal{E}_{2}$ is satisfied.

Note that in order to prove Proposition 3.6 it is enough to find $Z\in\overline{\mathcal{K}_{\ast}\left(X_{0}\right)}\setminus\left\{0\right\}$ such that

[TABLE]

because by definition of the closure and the continuity of $\mathcal{A}$ this implies that there is a matrix $\tilde{Z}\in\mathcal{K}_{\ast}\left(X_{0}\right)\setminus\left\{0\right\}$ such that

[TABLE]

which implies (13) with constant $C_{3}=8$. In the following we will construct such a matrix $Z$. Let $x\in\mathbb{R}^{r}$ be a vector such that $\|x\|=1$. Then for $a\in\left[n_{1}\right]$ as above we define the vector $w_{a}\in\mathbb{R}^{n_{2}}$ by

[TABLE]

and set

[TABLE]

It follows directly from the definition of $\mathcal{N}_{a}$ that $\mathcal{A}\left(W\right)=0$. In the following let $T$ be the tangent space of the manifold of rank-$r$ matrices at $X_{0}$ as defined in (14). Furthermore, denote by $P_{U}=UU^{*}$ the orthogonal projection onto the column space of $U$ and, analogously, by $P_{V}=VV^{*}$ the orthogonal projection onto the column space of $V$. Then we obtain that

[TABLE]

where in the second equality we have used that $\mathcal{P}_{T^{\perp}}M=P_{U^{\perp}}MP_{V^{\perp}}$ for all $M\in\mathbb{R}^{n_{1}\times n_{2}}$ and in the last line we used that $\|P_{U^{\perp}}e_{i}\|\leq 1$. Plugging in $w_{a}=P_{\mathcal{N}^{\perp}_{a}}Vx$ it follows that

[TABLE]

where the last line is due to $P_{V^{\perp}}Vx=0$. The fact that $\|P_{V^{\perp}}\|\leq 1$ then yields that

[TABLE]

where for the last line we used that $a\in\left[n_{1}\right]$ was chosen such that the condition in $\mathcal{E}_{2}$ holds. This shows that $W$ is relatively close to $T$. Based on $W$ we now aim to find $Z\in\overline{\mathcal{K}_{\ast}\left(X\right)}$ of the form

[TABLE]

where $\beta>0$ will be chosen in the following such that $Z\in\overline{\mathcal{K}_{\ast}\left(X_{0}\right)}$, which by Lemma 4.1 is equivalent to $-\langle UV^{*},Z\rangle_{F}\geq\|\mathcal{P}_{T^{\perp}}Z\|_{\ast}$ . First, we note that

[TABLE]

due to $\mathcal{P}_{T^{\perp}}\left(UV^{*}\right)=0$ and the inequality chain (31). Furthermore, we have that

[TABLE]

Hence, setting $\beta=2\sqrt{\frac{m}{r^{2}n_{1}n_{2}}}$ and combining (32) and (33) it follows that $Z\in\overline{\mathcal{K}_{\ast}\left(X_{0}\right)}$. This $Z$ also satisfies (30). To see that we observe that

[TABLE]

where in the last line we used the assumption that $m\leq\frac{n_{1}n_{2}}{32}$. Furthermore, from $\mathcal{A}\left(W\right)=0$ it follows that

[TABLE]

Combining the last two inequality chains implies (30), which completes the proof. ∎

4.4 Proof of Theorem 3.1 and Theorem 3.5

As already mentioned Theorem 3.1 can be deduced from Proposition 3.3 and Theorem 3.5 can be deduced from Proposition 3.6. We only show how to prove Theorem 3.1, as the proof of Theorem 3.5 is analogous.

Proof of Theorem 3.1.

By Proposition 3.3 and the definition of the minimum conic singular value $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\right)$ with probability at least $1-\mathcal{O}\left(\exp\left(-\frac{K}{C_{2}\mu^{2}}\right)\right)$ there is a matrix $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\setminus\left\{0\right\}$ such that

[TABLE]

and such that $\tilde{X}_{t}:=h_{0}m^{*}_{0}+tZ$ obeys $\|\tilde{X}_{t}\|_{\ast}\leq\|h_{0}m^{*}_{0}\|_{\ast}$ for all $0<t\leq 1$. Next, set $e_{t}=\frac{t}{2}\mathcal{A}\left(Z\right)$. Then for $y_{t}=\mathcal{A}\left(h_{0}m^{*}_{0}\right)+e_{t}$ we have that

[TABLE]

Hence, by setting $\tau_{0}:=\frac{\|\mathcal{A}\left(Z\right)\|}{2}$ we observe that $\|\mathcal{A}\left(\hat{X}_{t}\right)-y_{t}\|=t\tau_{0}$. Furthermore, note that

[TABLE]

Now let $0<\tau\leq\tau_{0}$. Then by setting $t=\frac{\tau}{\tau_{0}}$, $\tilde{X}:=\tilde{X}_{t}$, $y:=y_{t}$, and $e:=e_{t}$ the desired claim follows. ∎

5 Stability of blind deconvolution

5.1 Outline of the proof and main ideas

The goal of this section is to prove Theorem 3.7. We first give a proof sketch and present the key ideas. We have seen in Proposition 3.3 that for certain isometries $B\in\mathbb{C}^{L\times K}$ with high probability one has that $\lambda_{\min}\left(\mathcal{A},\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\right)\lesssim\sqrt{\frac{L}{KN}}$. Hence, applying Theorem 2.4 cannot lead to very strong error estimates. However, if we closely inspect the proof of Proposition 3.3 we observe the following. Again, denote by $T$ the tangent space of the manifold of rank-$1$ matrices at point $h_{0}m^{*}_{0}$ as defined in (14) and assume that $\|h_{0}\|=\|m_{0}\|=1$. By construction we have that $Z=W-\beta h_{0}m^{*}_{0}$, where $W=\frac{\langle h_{0},e_{i}\rangle}{\|m_{i}^{\perp}\||\langle h_{0},e_{i}\rangle|}e_{i}\left(m_{i}^{\perp}\right)^{*}$. This implies that

[TABLE]

where we have used the triangle inequality in the first line and the definition of $m^{\perp}_{i}$ in the second line. In the third line we used that $\|m^{\perp}_{i}\|\leq 1$, $|\langle e_{i},h_{0}\rangle|\leq\frac{\mu}{\sqrt{L}}$, and $\beta=\sqrt{\frac{L}{KN}}$. As $\|Z\|_{F}\gtrsim 1$ this implies that $\frac{\langle Z,h_{0}m^{*}_{0}\rangle_{F}}{\|Z\|_{F}\|h_{0}m^{*}_{0}\|_{F}}$ is quite small, meaning that $Z$ and $h_{0}m^{*}_{0}$ are almost orthogonal to each other. All the descent directions $Z$ with this property, however, have in common that the admissible descent step size

[TABLE]

is necessarily very small. Geometrically this corresponds to the fact that the nuclear norm ball is curved near $X_{0}$, which is why its near-tangential behaviour only holds locally. This will be made precise in Lemma 5.7 below. For this reason the idea of the proof of Theorem 3.7 is to split the descent cone into two parts. One part will consist of all the matrices aligned with $h_{0}m^{*}_{0}$. The second part will consist of all remaining matrices, which are almost orthogonal to $h_{0}m^{*}_{0}$. As mentioned above, these matrices must necessarily be close to $T$. The first part is captured by the set $\mathcal{E}_{\mu,\delta}$ with

[TABLE]

where $\delta>0$. In Section 5.2 we will show that with high probability it holds that

[TABLE]

Hence, if we have for the minimizer $\hat{X}$ of (3) that $\hat{X}-h_{0}m^{*}_{0}$ is an element of the conic hull of $\mathcal{E}_{\mu,\delta}$ we can proceed similarly as in [13] to obtain near-optimal error bounds. Let us briefly explain which property of $\mathcal{E}_{\mu,\delta}$ allows us to show (35). We define for any matrix $W\in\mathbb{C}^{K\times N}$ its $\|\cdot\|_{B_{1}}$-norm by

[TABLE]

In other words $\|W\|_{B_{1}}$ is the $\ell_{1}$-norm of the vector $\left(\|W^{*}b_{\ell}\|\right)^{L}_{\ell=1}$. We show in Lemma 5.5 below that all $Z\in\mathcal{E}_{\mu,\delta}$ have rather large $\|\cdot\|_{B_{1}}$-norm, which entails that the mass of the vector $\left(\|W^{*}b_{\ell}\|\right)^{L}_{\ell=1}$ cannot be concentrated on only very few entries. This in turn will allow us to employ a non-i.i.d. version of Mendelson’s small ball method [37], allowing us to show a lower bound for (35), see Lemma 5.6 below. To understand the behaviour on the second part recall from Proposition 3.3 that for matrices $Z/\|Z\|_{F}\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\setminus\mathcal{E}_{\mu,\delta}$ the quantity $\|\mathcal{A}\left(Z\right)\|$ may be quite small, so a uniform bound is not feasible. However, as $Z$ is almost orthogonal to $h_{0}m^{*}_{0}$, also $\|\mathcal{P}_{T^{\perp}}Z\|_{\ast}$ must be rather small because of the characterization of the descent cone, Lemma 4.1. Hence, $Z$ is close to the tangent space and is almost orthogonal to $h_{0}m^{*}_{0}$. For that reason, whenever $\|h_{0}m^{*}_{0}+tZ\|_{\ast}\leq\|h_{0}m^{*}_{0}\|_{\ast}$ holds, the cylindrical shape of the nuclear norm ball implies that $t>0$ is small. This fact is captured by Lemma 5.7 below. Theorem 3.3 can then be proven by combining inequality (35) and Lemma 5.7, see Section 5.3.

5.2 A lower bound for the minimum conic singular value

First we recall the notion of Gaussian width (see, e.g., [63]).

Definition 5.1**.**

For a set $\mathcal{E}\subset\mathbb{C}^{K\times N}$ its Gaussian width is defined by

[TABLE]

where $G\in\mathbb{C}^{K\times N}$ is a matrix, whose entries are independent and identically distributed random variables with distribution $\mathcal{CN}\left(0,1\right)$.

This definition allows us to state the following lemma, which is important for our analysis of the conic singular value. It relies on a uniform lower bound on the number of measurements whose magnitude is larger than a certain constant and is a variant of Theorem 2.1 in [37].

Lemma 5.2**.**

Let $\mathcal{E}\subset\mathbb{C}^{K\times N}$ be a symmetric set, i.e., $\mathcal{E}=-\mathcal{E}$. For all $\xi>0$ and $t>0$ it holds with probability at least $1-\exp\left(-2t^{2}\right)$ that

[TABLE]

Here $\varepsilon_{1},\ldots,\varepsilon_{L}$ are independent Rademacher variables, i.e., random variables which take the two values $\pm 1$ each with probability $\frac{1}{2}$.

The proof of the Lemma 5.2 is based on a variant of Mendelson’s small-ball method and proceeds in analogy to [37]. We have deferred a detailed proof to Appendix A. In order to apply Lemma 5.2 we need to estimate the first term of the right-hand side of Lemma 5.2. Such an estimate can be derived using the Paley-Zygmund inequality as in [37]. For the sake of completeness we have included a proof in Appendix B.

Lemma 5.3**.**

Let $X\in\mathbb{C}^{K\times N}$ be an arbitrary matrix. Then for all $\xi>0$

[TABLE]

In order to use Lemma 5.3 we need a lower bound for $|\left\{\ell\in\left[L\right]:\ \|X^{*}b_{\ell}\|\geq\xi\right\}|$. This will be achieved by the next lemma. For the statement of this lemma and its proof we will need to introduce the following notion. We define for any matrix $W\in\mathbb{C}^{K\times N}$ its $\|\cdot\|_{B_{1,w}}$-quasinorm by

[TABLE]

That is, $\|W\|_{B_{1,w}}$ is the weak $\ell_{1}$-norm of the vector $\left(\|W^{*}b_{\ell}\|\right)^{L}_{\ell=1}$. (For a more detailed discussion of the weak $\ell_{1}$-norm see, e.g., [24].) A direct consequence of this interpretation is the inequality (see, e.g., [22, Proposition 2.10 and Exercise 2.4])

[TABLE]

Lemma 5.4**.**

Let $Z\in\mathbb{C}^{K\times N}$ such that $\|Z\|_{F}=1$. Then it holds that

[TABLE]

Proof of Lemma 5.4.

Choose $\xi^{*}$ such that

[TABLE]

As $|\left\{\ell\in\left[L\right]:\ \|Z^{*}b_{\ell}\|\geq\xi^{*}\right\}|\leq L$ it follows that

[TABLE]

where we also used inequality (37). We observe that

[TABLE]

where for the second equality we used the identity $\sum_{\ell=1}^{L}b_{\ell}b^{*}_{\ell}=\text{Id}$. Using inequality (39) and rearranging terms it follows that

[TABLE]

which completes the proof.

∎

The next lemma gives a bound on $\underset{Z\in\mathcal{E}_{\mu,\delta}}{\inf}\|Z\|_{B_{1}}$.

Lemma 5.5**.**

It holds that

[TABLE]

Proof.

Let $Z\in\mathcal{E}_{\mu,\delta}$ be arbitrary. By definition of $\mathcal{E}_{\mu,\delta}$ there is $h_{0}\in\mathcal{H}_{\mu}$ and $m_{0}\in\mathbb{C}^{N}$ such that $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$ and such that the inequality

[TABLE]

holds. It follows that

[TABLE]

where for the second equality we have used that $\sum_{\ell=1}^{L}b_{\ell}b^{*}_{\ell}=\text{Id}$. Note that for all $\ell\in\left[L\right]$ it holds that

[TABLE]

Hence, by the previous inequality chain it follows that

[TABLE]

where in the last inequality we used the definition of $\mu$ and $\|Z\|_{B_{1}}$. Rearranging terms and taking the infimum over all $Z\in\mathcal{E}_{\mu,\delta}$ yields the desired inequality. ∎

Having gathered all the necessary ingredients we can state and prove the main lemma of this section.

Lemma 5.6**.**

Let $\delta>0$. Assume that

[TABLE]

Then with probability at least $1-\exp\left(-\frac{L\delta^{4}}{C_{2}\log^{4}\left(eL\right)\mu^{4}}\right)$ it holds that

[TABLE]

$C_{1}$* and $C_{2}$ are absolute constants.*

Proof.

Our goal is to apply Lemma 5.2. In order to apply it we first derive a lower bound for the first term on the right-hand side of inequality (36). For that recall that by Lemma 5.5 it holds that

[TABLE]

Thus, for any $Z\in\mathcal{E}_{\mu,\delta}$ we obtain that

[TABLE]

where the first inequality follows from (42), the second one is due to Lemma 5.4, and the third one follows again from (42). Hence, by Lemma 5.3 applied with $\xi=\frac{\delta}{4\sqrt{L}\ln\left(eL\right)\mu}$ we finally obtain that

[TABLE]

Next, we need an upper bound for the Gaussian width. For that, we first observe that

[TABLE]

The Gaussian width of $\mathcal{E}$ has been bounded in [31, Lemma 4.1], combined with the monotonicity of the Gaussian width their results yields that

[TABLE]

Thus for $\xi=\frac{\delta}{4\sqrt{L}\log\left(eL\right)\mu}$ we obtain from Lemma 5.2 together with (43), (44) that with probability at least $1-\exp\left(-2t^{2}\right)$ it holds that

[TABLE]

where the second inequality follows from assumption (40), if the constant $C_{1}>0$ is chosen large enough. Consequently, setting $t=\frac{9\delta^{2}\sqrt{L}}{128\log^{2}\left(eL\right)\mu^{2}}$ and recalling that $\left(\mathcal{A}\left(Z\right)\right)\left(\ell\right)=\langle b_{\ell}c_{\ell}^{*},Z\rangle_{F}$ we have that with probability at least $1-\exp\left(-\frac{L\delta^{4}}{C_{2}\log^{4}\left(eL\right)\mu^{4}}\right)$ with $C_{2}$ chosen appropriately

[TABLE]

Summing up we obtain that with probability at least $1-\exp\left(-\frac{L\delta^{4}}{C_{2}\log^{4}\left(eL\right)\mu^{4}}\right)$

[TABLE]

This shows the claim. ∎

5.3 Proof of Theorem 3.7

As already mentioned in Section 4, in order to control all matrices $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$, which are almost orthogonal to $h_{0}m^{*}_{0}$, we need the following key lemma.

Lemma 5.7**.**

Let $h_{0}m^{*}_{0}\in\mathbb{C}^{n_{1}\times n_{2}}$ be a rank-1 matrix. Assume that $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)\setminus\left\{0\right\}$. Then, whenever $\|h_{0}m^{*}_{0}+Z\|_{\ast}\leq\|h_{0}m^{*}_{0}\|_{\ast}$, it holds that

[TABLE]

Proof.

We observe that

[TABLE]

Rearranging terms yields the result. ∎

Now we have gathered all tools which are needed to prove Theorem 3.7.

Proof of Theorem 3.7.

Having introduced all necessary tools in the last two sections we can now give a proof of Theorem 3.7. We set $\delta:=\left(\log eL\right)^{2/3}\mu^{2/3}\alpha^{1/3}$. Throughout the proof we will assume that inequality (41) holds, which by Lemma 5.6 holds with probability at least

[TABLE]

Let $h_{0}\in\mathcal{H}_{\mu}$ and $m_{0}\in\mathbb{C}^{N}$. Furthermore, let $\hat{X}$ be a minimizer of (3) and set $Z:=\hat{X}-h_{0}m^{*}_{0}$. Note that from the minimality of $\hat{X}$ it follows that $\|\hat{X}\|_{\ast}\leq\|h_{0}m^{*}_{0}\|_{\ast}$. This implies that $Z\in\mathcal{K}_{\ast}\left(h_{0}m^{*}_{0}\right)$. To prove the lemma it remains to derive an appropriate upper bound on $\|Z\|_{F}$. For that we will distinguish two cases, namely $\frac{Z}{\|Z\|_{F}}\in\mathcal{E}_{\mu,\delta}$ and $\frac{Z}{\|Z\|_{F}}\notin\mathcal{E}_{\mu,\delta}$. If $\frac{Z}{\|Z\|_{F}}\in\mathcal{E}_{\mu,\delta}$, it follows from inequality (41) that

[TABLE]

where in the second inequality we used the triangle inequality as well as $Z=\hat{X}-h_{0}m^{*}_{0}$ and $y=\mathcal{A}\left(h_{0}m^{*}_{0}\right)+e$. In the third inequality we used that $\hat{X}$ is feasible and $\|e\|\leq\tau$. If $\frac{Z}{\|Z\|_{F}}\notin\mathcal{E}_{\mu,\delta}$, it follows directly from the definition of $\mathcal{E}_{\mu,\delta}$ that $-\text{Re}\left(\langle\frac{h_{0}m^{*}_{0}}{\|h_{0}m^{*}_{0}\|_{F}},\frac{Z}{\|Z\|_{F}}\rangle_{F}\right)<\delta$. By Lemma 5.7 we obtain that

[TABLE]

Combining the estimates (45) and (46) we obtain that

[TABLE]

which completes the proof. ∎

6 Outlook

In this paper we have analyzed two important cases of structured low-rank matrix recovery problems, blind deconvolution and matrix completion, through an inspection of the descent cone of the nuclear norm and its interaction with the measurement operator $\mathcal{A}$. We have shown that the conic singular value is typically quite small and, consequently, previous analysis approaches cannot give strong recovery guarantees. For the example of blind deconvolution we have presented a new approach based on a refined analysis of the descent cone, showing that the nuclear norm minimization approach is stable against adversarial noise in certain important parameter regimes and allows for uniform recovery guarantees in the presence of noise. In our opinion our results give rise to a number of interesting follow-up questions.

• •

**Stability for small noise-levels: ** Until now, our stability result only covers the situation that the noise level $\tau$ is of constant order (up to logarithmic factors). For small $\tau$, Theorem 3.1, respectively Theorem 3.5, put some barriers on what performance can be expected. Nevertheless, it will be interesting to examine the transitional case, that $\tau$ is rather small, even further. For example, while the bad conditioning for small noise levels has been established, it remains open whether one can construct a noise vector $e$ such that the true minimizer behaves like the alternative (but non-minimal) solutions constructed in Theorems 3.1 and 3.5. Also the transitional behavior of the minimum conic singular values between very small noise levels (where we established bad conditioning) and larger noise levels (where at least for randomized blind deconvolution, we proved stability) will be an interesting question to study.

• •

Extension to the rank $r$ case: Understanding nuclear norm recovery for matrix completion under adversarial noise remains an important open problem in the field. While our result established that recovery guarantees for arbitrary noise levels are not feasible, our considerations for the rank one scenario give hope that for sufficiently large noise levels, near optimal guarantees are within reach also for matrices of arbitrary rank.

Similarly, a natural generalization of blind deconvolution is the problem of blind demixing [46, 30], where one observes a noisy superposition of several convolutions, that is, $y=\sum_{i=1}^{r}w_{i}\ast x_{i}+e$. The corresponding low-rank matrix formulation can be interpreted as a rank $r$ version of the randomized blind deconvolution problem.

We expect that a rank $r$ version of Theorem 3.7 will apply to both these scenarios, which is why we consider this a very promising direction for future research.

• •

Extension to other low-rank matrix recovery models: Various other low-rank matrix models also involve incoherence in some way, for example, robust PCA ([6]) and spectral compressed sensing via matrix completion [15]. Also for these problems, recovery results are typically proven via the Golfing Scheme and lead to a seemingly suboptimal noise bound (see, e.g., [65, Section VI]). Can these problems be analyzed with the methods developed in this paper?

Moreover, [38] provided an incoherence based analysis of the phase retrieval problem under random Bernoulli measurements. It will be interesting to analyze this setup with similar methods as in this manuscript.

Acknowledgements

The authors would like to thank Richard Kueng for discussions related to this topic, which sparked their interest in the problem. Furthermore, the authors want to thank Peter Jung, Marius Junge, and Kiryung Lee for fruitful discussions. This work has been supported by the German Science Foundation (DFG) in the context of the project Bilinear Compressed Sensing (KR 4512/2-1) as a part of the Priority Program 1798, and the Emmy-Noether Junior Research Group Randomized Sensing and Quantization of Signals and Images (KR 4512/1-1).

Appendix A Proof of Lemma 5.2

The proof of Lemma 5.2 will rely on the following two lemmas. The first lemma is a version of Mendelson’s small-ball method for non-i.i.d. measurements. In order to state it let $X_{1},\ldots,X_{L}$ be independent, matrix-valued random variables defined on a probability space $\left(\Omega,\mu\right)$. For every measurable, real-valued function $f$ and for every $\xi>0$ we define the quantity

[TABLE]

Lemma A.1**.**

Let $X_{1},\ldots,X_{L}\in\mathbb{C}^{K\times N}$ be independent random variables and $\mathcal{F}$ be a set of real-valued functions, which are measurable with respect to $\left(\Omega,\mu\right)$. Let $t>0$ and $\xi>0$. Then with probability at least $1-\exp\left(-2t^{2}\right)$ it holds that

[TABLE]

where $\varepsilon_{1},\ldots,\varepsilon_{L}$ are independent Rademacher variables, i.e., random variables which take the two values $\pm 1$ each with probability $\frac{1}{2}$.

The proof of Lemma A.1 is exactly analogous as the proof of the original small-ball method [37], see Section A.1. The second auxiliary lemma, proved in Section A.2, relates the quantity $\mathbb{E}\left[\underset{f\in\mathcal{F}}{\sup}\sum_{\ell=1}^{L}\varepsilon_{\ell}f\left(X_{\ell}\right)\right]$ in the blind deconvolution framework to the Gaussian width (cf. Definition 5.1).

Lemma A.2**.**

Let $\mathcal{E}\subset\mathbb{C}^{K\times N}$. Then it holds that

[TABLE]

With these lemmas we can now prove Lemma 5.2.

Proof of Lemma 5.2.

Set $X_{\ell}:=b_{\ell}c^{*}_{\ell}$ for all $\ell\in\left[L\right]$ and define

[TABLE]

Then, by a direct application of Lemma A.1 we obtain that with probability at least $1-\exp\left(-2t^{2}\right)$ it holds that

[TABLE]

To bound the second summand, we observe that

[TABLE]

where in the third line we used that $\text{Re}\left(\langle b_{\ell}c^{*}_{\ell},X\rangle_{F}\right)$ and $\text{Im}\left(\langle b_{\ell}c^{*}_{\ell},X\rangle_{F}\right)$ have the same distribution. The fourth line follows from the symmetry of the set $\mathcal{E}$ and the last line is due to Lemma A.2. Combining (47) and (48) finishes the proof. ∎

A.1 Proof of Lemma A.1

We directly trace the steps of the proof of Theorem 1.5 in [37]. In the following $\mathbbm{1}_{A}$ denotes the indicator function, which takes the value $1$, if the event $A$ occurs and the value [math] otherwise. Note that

[TABLE]

Taking the infimum we observe that by the definition of $Q_{2\xi}$

[TABLE]

The bounded difference inequality (see, for example, [4]) implies that with probability at least $1-\exp\left(-2t^{2}\right)$ it holds that

[TABLE]

To deal with the expectation we will use the function $\Psi_{\xi}:[0,+\infty)\longrightarrow\mathbb{R}$ defined by

[TABLE]

We observe that $\Psi_{\xi}$ is Lipschitz continuous with Lipschitz constant $1/\xi$. Furthermore, for all $u\in[0,+\infty)$ it holds that $\mathbbm{1}_{\left\{u\geq 2\xi\right\}}\leq\Psi_{\xi}\left(u\right)\leq\mathbbm{1}_{\left\{u\geq\xi\right\}}$. Combining this monotonicity relation with Gine-Zinn symmetrization (see, e.g., [62, Lemma 2.3.1]) and the Rademacher comparison principle for Lipschitz continuous functions (see, e.g., [44, Equation (4.20)]), we obtain that

[TABLE]

Together with the inequality chains (49) and (50), this completes the proof. ∎

A.2 Proof of Lemma A.2

First, we observe that

[TABLE]

Note that due to the definition of $\mathcal{\omega}\left(\mathcal{E}\right)$ in order to finish the proof it is enough to show that the entries of the matrix $X=\sum_{\ell=1}^{L}b_{\ell}c^{*}_{\ell}$ are independent and identically distributed with distribution $\mathcal{CN}\left(0,1\right)$. For that, let $\left(i,j\right)\in\left[K\right]\times\left[N\right]$ and compute that

[TABLE]

This implies that $e^{*}_{i}\left(\sum_{\ell=1}^{L}b_{\ell}c^{*}_{\ell}\right)e_{j}\in\mathcal{CN}\left(0,1\right)$. It remains to show that the individual entries of the matrix $\sum_{\ell=1}^{m}b_{\ell}c^{*}_{\ell}$ are independent. For that, we set

[TABLE]

Now let $\left(i,j\right),\left(i^{\prime},j^{\prime}\right)\in\left[K\right]\times\left[N\right]$ such that $\left(i,j\right)\neq\left(i^{\prime},j^{\prime}\right)$. Our goal is to show that $\mathbb{E}\left[X_{i,j}\overline{X}_{i^{\prime},j^{\prime}}\right]=0$. If $j\neq j^{\prime}$ this follows immediately from the observation that $c^{*}_{\ell}e_{j}$ and $c^{*}_{\ell}e_{j^{\prime}}$ are independent for all $\ell\in\left[L\right]$. Now assume that $j=j^{\prime}$. Then we can compute that

[TABLE]

Hence, we have shown that all entries of the matrix $X$ are uncorrelated. As the entries of $X$ are jointly Gaussian this implies that they are independent, which completes the proof.

∎

Appendix B Proof of Lemma 5.3

Proof of Lemma 5.3.

Let $\ell\in\left[L\right]$ such that $\|X^{*}b_{\ell}\|\geq 4\xi$. Using the Paley-Zygmund inequality (see, e.g., [18]) we obtain that

[TABLE]

(We used that $\mathbb{E}|\langle b_{\ell}c_{\ell}^{*},X\rangle_{F}|^{2}=\|X^{*}b_{\ell}\|^{2}$ and $\mathbb{E}|\langle b_{\ell}c_{\ell}^{*},X\rangle_{F}|^{4}=2\|X^{*}b_{\ell}\|^{4}$, which is due to $\langle b_{\ell}c_{\ell}^{*},X\rangle_{F}\sim\mathcal{CN}\left(0,\|X^{*}b_{\ell}\|\right)$.) Summing over all $\ell\in\left[L\right]$ such that $\|X^{*}b_{\ell}\|\geq 4\xi$ yields the claim. ∎