Low-rank matrix recovery via regularized nuclear norm minimization

TL;DR

This paper provides a theoretical analysis of low-rank matrix recovery using regularized nuclear norm minimization, establishing conditions under which robust recovery from noisy measurements is guaranteed, and introduces new coefficient estimates for null space properties.

Contribution

It is the first to establish $tk$-order RIC based coefficient estimates for the null space property in the case of $0<t\, extless=1$, extending recovery guarantees.

Findings

Recovery condition matches previous sharp bounds for $t>4/3$

Robust recovery is possible with noisy measurements under certain RIC constraints

First to analyze coefficient estimates for null space property when $0<t extless=1$

Abstract

In this paper, we theoretically investigate the low-rank matrix recovery problem in the context of the unconstrained regularized nuclear norm minimization (RNNM) framework. Our theoretical findings show that, the RNNM method is able to provide a robust recovery of any matrix $X$ (not necessary to be exactly low-rank) from its few noisy measurements $\textbf{b}=\mathcal{A}(X)+\textbf{n}$ with a bounded constraint $\|\textbf{n}\|_{2}\leq\epsilon$, provided that the $tk$-order restricted isometry constant (RIC) of $\mathcal{A}$ satisfies a certain constraint related to $t>0$. Specifically, the obtained recovery condition in the case of $t>4/3$ is found to be same with the sharp condition established previously by Cai and Zhang (2014) to guarantee the exact recovery of any rank-$k$ matrix via the constrained nuclear norm minimization method. More importantly, to the best of our knowledge, we are the first to establish the $tk$-order RIC based coefficient estimate of the robust null space property in the case of $0<t\leq1$.

Equations

$$\mathbf{b}=\mathcal{A}(X)+\mathbf{n},$$

$$\mathcal{A}(X)=[\text{tr}(X^T{A}^{(1)}),\text{tr}(X^T{A}^{(2)}),\cdots ,\text{tr}(X^T{A}^{(m)}){]}^T.$$

$$\underset{X\in {\mathbb{R}}^{n_1\times n_2}}{\min }\Vert X{\Vert }_{\ast },s.t.\Vert \mathbf{b}-\mathcal{A}(X){\Vert }_2\le ϵ.$$

$$\Vert X^{♯}-X{\Vert }_F$$

$$(1-\delta )\Vert X{\Vert }_F^2\le \Vert \mathcal{A}(X){\Vert }_2^2\le (1+\delta )\Vert X{\Vert }_F^2$$

$${\delta }_{tk}<\{\begin{array}{rl}& \frac{t}{4-t},0<t\le \frac{4}{3},\\ & \sqrt{\frac{t-1}{t}},\frac{4}{3}<t<1.\end{array}$$

$$\underset{X\in {\mathbb{R}}^{n_1\times n_2}}{\min }\Vert X{\Vert }_{\ast }+\frac{1}{2\lambda }\Vert \mathbf{b}-\mathcal{A}(X){\Vert }_2^2,$$

$$H=\sum _{i=1}^{n_1}{\sigma }_i(H){\mathbf{a}}_H^{(i)}{\left({\mathbf{c}}_H^{(i)}\right)}^T,$$

$$T(\alpha ,k)$$

$$U(\alpha ,k,\mathbf{y})$$

$$\mathbf{v}=\sum _l{\gamma }_l{\mathbf{z}}^{(l)}$$

$$\sum _l{\gamma }_l\Vert {\mathbf{z}}^{(l)}{\Vert }_2^2\le k{\alpha }^2.$$

$${\delta }_{tk}<\sqrt{\frac{t-1}{t}}$$

$$\Vert H_{\Omega }{\Vert }_F\le {\beta }_1\Vert \mathcal{A}(H){\Vert }_2+{\beta }_2\frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }}{\sqrt{k}},$$

$${\beta }_1=\frac{2}{(1-{\delta }_{tk})\sqrt{1+{\delta }_{tk}}},\text{ and }{\beta }_2=\frac{{\delta }_{tk}}{\sqrt{(1-({\delta }_{tk}{)}^2)(t-1)}}.$$

$${\Lambda }_1=\left\{i\in {\Omega }^c:{\sigma }_i>\frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }}{(t-1)k}\right\},{\Lambda }_2=\left\{i\in {\Omega }^c:{\sigma }_i\le \frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }}{(t-1)k}\right\}.$$

$$\Vert H_{\Omega \cup {\Lambda }_1}{\Vert }_F\le {\beta }_1\Vert \mathcal{A}(H){\Vert }_2+\frac{{\beta }_2}{\sqrt{k}}\Vert H_{{\Omega }^c}{\Vert }_{\ast }$$

$$\Vert {\sigma }_{{\Lambda }_1}{\Vert }_1=\Vert H_{{\Lambda }_1}{\Vert }_{\ast }>|{\Lambda }_1|\frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }}{(t-1)k}\ge \frac{|{\Lambda }_1|}{(t-1)k}\Vert H_{{\Lambda }_1}{\Vert }_{\ast }=\frac{|{\Lambda }_1|}{(t-1)k}\Vert {\sigma }_{{\Lambda }_1}{\Vert }_1.$$

$$\Vert {\sigma }_{{\Lambda }_2}{\Vert }_1$$

$$\Vert {\sigma }_{{\Lambda }_2}{\Vert }_{\infty }$$

$${\sigma }_{{\Lambda }_2}=\sum _l{\gamma }_l{\mathbf{z}}^{(l)},$$

$$\sum _l{\gamma }_l\Vert {\mathbf{z}}^{(l)}{\Vert }_2^2\le ((t-1)k-|{\Lambda }_1|)\frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }^2}{(t-1{)}^2{k}^2}\le \frac{\Vert H_{{\Omega }^c}{\Vert }_{\ast }^2}{(t-1)k}.$$

$$B^{(l)}=(1+{\delta }_{tk})H_{\Omega \cup {\Lambda }_1}+{\delta }_{tk}{Z}^{(l)},D^{(l)}=(1-{\delta }_{tk})H_{\Omega \cup {\Lambda }_1}-{\delta }_{tk}{Z}^{(l)},$$

$$\displaystyle\rho\triangleq\sum_{l}\gamma_{l}\bigg{(}\|\mathcal{A}(B^{(l)})\|_{2}^{2}-\|\mathcal{A}(D^{(l)})\|_{2}^{2}\bigg{)}.$$

$$\rho$$

$$=4{\delta }_{tk}⟨\mathcal{A}(H_{\Omega \cup {\Lambda }_1}),\mathcal{A}(H)⟩\le 4{\delta }_{tk}\Vert \mathcal{A}(H_{\Omega \cup {\Lambda }_1}){\Vert }_2\Vert \mathcal{A}(H){\Vert }_2$$

$$\le 4{\delta }_{tk}\sqrt{1+{\delta }_{tk}}\Vert H_{\Omega \cup {\Lambda }_1}{\Vert }_F\Vert \mathcal{A}(H){\Vert }_2,$$

$$\rho$$

$$=2{\delta }_{tk}(1-({\delta }_{tk}{)}^2)\Vert {\sigma }_{\Omega \cup {\Lambda }_1}{\Vert }_2^2-2({\delta }_{tk}{)}^3\sum _l{\gamma }_l\Vert {\mathbf{z}}^{(l)}{\Vert }_2^2$$

$$\ge 2{\delta }_{tk}(1-({\delta }_{tk}{)}^2)\Vert H_{\Omega \cup {\Lambda }_1}{\Vert }_F^2-\frac{2({\delta }_{tk}{)}^3}{(t-1)k}\Vert H_{{\Omega }^c}{\Vert }_{\ast }^2,$$

$$(1-({\delta }_{tk}{)}^2)\Vert H_{\Omega \cup {\Lambda }_1}{\Vert }_F^2-2\sqrt{1+{\delta }_{tk}}\Vert \mathcal{A}(H){\Vert }_2\Vert H_{\Omega \cup {\Lambda }_1}{\Vert }_F-\frac{({\delta }_{tk}{)}^2}{(t-1)k}\Vert H_{{\Omega }^c}{\Vert }_{\ast }^2\le 0.$$

$$\Vert H_{E\cup E_1}{\Vert }_F\le$$

$$+\frac{\sqrt{(2\sqrt{1+{\delta }_{tk}}\Vert \mathcal{A}(H){\Vert }_2{)}^2+4(1-({\delta }_{tk}{)}^2)\frac{({\delta }_{tk}{)}^2}{(t-1)k}\Vert H_{E^c}{\Vert }_{\ast }^2}}{2(1-({\delta }_{tk}{)}^2)}$$

Full text

Low-rank matrix recovery via regularized nuclear norm minimization111Email addresses: wdwang@swu.edu.cn (Wendong Wang), zhangf@email.swu.edu.cn (Feng Zhang), wjj@swu.edu.cn (Jianjun Wang). The corresponding author is Jianjun Wang.

Wendong Wang1, Feng Zhang2, and Jianjun Wang2

1. College of Artificial Intelligence, Southwest University, Chongqing, 400715, China

2. School of Mathematics and Statistics, Southwest University, Chongqing, 400715, China

Abstract.

In this paper, we theoretically investigate the low-rank matrix recovery problem in the context of the unconstrained regularized nuclear norm minimization (RNNM) framework. Our theoretical findings show that, the RNNM method is able to provide a robust recovery of any matrix $X$ (not necessary to be exactly low-rank) from its few noisy measurements $\bm{b}=\mathcal{A}(X)+\bm{n}$ with a bounded constraint $\|\bm{n}\|_{2}\leq\epsilon$, provided that the $tk$-order restricted isometry constant (RIC) of $\mathcal{A}$ satisfies a certain constraint related to $t>0$. Specifically, the obtained recovery condition in the case of $t>4/3$ is found to be same with the sharp condition established previously by Cai and Zhang (2014) to guarantee the exact recovery of any rank-$k$ matrix via the constrained nuclear norm minimization method. More importantly, to the best of our knowledge, we are the first to establish the $tk$-order RIC based coefficient estimate of the robust null space property in the case of $0<t\leq 1$.

Key words.

Low-rank matrix recovery, regularized nuclear norm minimization, restricted isometry property, robust null space property

1 Introduction

Over the past decade, low-rank matrix recovery (LRMR) problem has attracted considerable interest of researchers in many fields, including computer vision [1], recommender systems [2], and machine learning [3], to name a few. Mathematically, this problem aims to recover an unknown low-rank matrix $X\in\mathbb{R}^{n_{1}\times n_{2}}$ from

[TABLE]

where $\bm{b}\in\mathbb{R}^{m}(m\ll n_{1}n_{2})$ is an observed vector, $\bm{n}\in\mathbb{R}^{m}$ is the unknown noise, and $\mathcal{A}:\mathbb{R}^{n_{1}\times n_{2}}\rightarrow\mathbb{R}^{m}$ is a known linear measurement map defined as

[TABLE]

Here, $A^{(i)}$ for $i=1,2,\cdots,m$ is denoted as a matrix with size $n_{1}\times n_{2}$, and $\text{tr}(\cdot)$ is the trace function.

A popular approach for the LRMR problem is to solve a convex nuclear norm minimization (NNM) model

[TABLE]

So far, much work has been done to explore the theoretical performance of (1.2) in exact/robust recovery of any matrix that is not necessary to be exactly low-rank, see, e.g., [4, 5, 6, 7, 8, 9, 10, 11, 12]. More specifically, one may seek the sufficient conditions under which the upper-bound estimate of the recovery error will take the form

[TABLE]

where $X^{\sharp}$ and $X_{[k]}$ are denoted by the optimal solution of (1.2) and the best rank-$k$ approximate of $X$, respectively, and $C_{1},C_{2}$ are two constants only related to the map $\mathcal{A}$. Note that (1.3) also indicates that under these conditions any rank-$k$ matrices, i.e., the matrices whose rank is at most $k$, can be exactly recovered from (1.2) provided that there is no noise involved, i.e., $\bm{n}$=0 and $\epsilon=0$. As one of the most powerful and widely used theoretical tools, restricted isometry property (RIP) captures particular attention in establishing these desired conditions and their resulting upper-bound estimates of the recovery error.

Definition 1.1** ([5]).**

A linear map $\mathcal{A}$ given in (1.1) is said to satisfy the RIP with restricted isometry constant (RIC) of order $k$, denoted by $\delta_{k}$222When $k$ is not an integer, we define $\delta_{k}$ as $\delta_{\lceil k\rceil}$., if $\delta_{k}$ is the smallest value $\delta\in(0,1)$ such

[TABLE]

holds for every rank-$k$ matrix $X\in\mathbb{R}^{n_{1}\times n_{2}}$.

Some representative conditions include $\delta_{4k}<0.558$ and $\delta_{3k}<0.4721$ in [13], $\delta_{2k}<0.4931$ and $\delta_{k}<0.309$ in [14], and $\delta_{2k}<1/2$ and $\delta_{k}<1/3$ in [8]. In particular, the sharp conditions for exactly rank-$k$ matrix recovery, which takes the form of $\delta_{tk}<\delta^{*}$, have been completely given by Cai and Zhang in [10] and Zhang and Li in [11] for the cases of $0<t<4/3$ and $0<t\leq 4/3$, respectively. To be specific, we can write these sharp conditions into a compact form as below,

[TABLE]

In fact, under the condition (1.4) any (nearly) low-rank matrix can still be robustly recovered from (1.2) in the presence of noise, and more details can be found within [10, 11].

Generally, when confronted with the relatively small problems where a high degree of numerical precision is required, one can easily formulate (1.2) as a semidefinite program (SDP), see, e,.g., [4, 5], and thus numerically solve it by any of the standard SDP solvers. However, when the scale of the input data is relatively large, it is often not convenient (sometimes maybe impossible) to solve (1.2) by any standard SDP solvers. Moreover, it is also difficult to estimate a proper parameter value of $\epsilon$ in (1.2) to well accommodate the unknown noise. Instead of solving (1.2) directly, many algorithms, see, e.g., [15, 16, 17, 18], were proposed to solve the following unconstrained regularized NNM (RNNM) model

[TABLE]

where $\lambda>0$ is a trade-off parameter. Compared with the constrained optimization problem (1.2), the unconstrained optimization problem (1.5) can well balance the low-rankness of the desired output matrix and the resultant recovery error with properly chosen values of parameter $\lambda$. It has been proved in the practical application that (1.5) is much more suitable for noisy measurements and approximately low-rank matrix recovery [18]. Nevertheless, one would hope that a result similar to (1.3) can be proved for (1.5) as well. To the best of our knowledge, Candès and Plan [5] gave the first RIP-based performance guarantee for (1.5), and their results show that, when the noise $\bm{n}$ obeys $\|\mathcal{A}^{*}(\bm{n})\|\triangleq\|\sum_{i=1}^{m}\bm{n}_{i}\cdot A^{(i)}\|\leq\lambda/2$, and the map $\mathcal{A}$ satisfies $\delta_{4k}<(3\sqrt{2}-1)/17$, the robust recovery of any rank-$k$ matrices can be guaranteed through (1.5). However, after their initial work, the theoretical investigation of (1.5) is rarely reported. Note that their noise setting is based on the Dantzig selector rather than the often used $\ell_{2}$-norm setting (i.e., $\|\bm{n}\|_{2}\leq\epsilon$), and the obtained sufficient condition still has room to improve.

In this paper, by means of the powerful RIP tool, we theoretically investigate the performance guarantees of the unconstrained RNNM model (1.5) when the noise $\bm{n}$ obeys $\|\bm{n}\|_{2}\leq\epsilon$. In summary, our contributions are two-fold. First, we show that if $\mathcal{A}$ obeys $\delta_{tk}<\sqrt{(t-1)/t}$ for certain $t>1$, then the unconstrained RNNM model (1.5) will be able to provide a robust matrix recovery performance. The obtained sufficient condition is in line with the sharp recovery condition (1.4) in the case of $t>4/3$ for the constrained problem (1.2). Second, by establishing the $tk$-order RIC based coefficient estimate of the robust null space property (RNSP) in the case of $0<t\leq 1$, we develop another $tk$-order RIC based sufficient condition for (1.5), and also obtain some new upper-bound estimates of recovery error.

The remainder of the paper is organized as follows. Section 2 introduces some necessary notations and lemmas. Section 3 presents a performance guarantee of the RNNM model (1.5) by means of the $tk$-order RIC with $t>1$. In Section 4, we first establish a $tk$-order RIC based coefficient estimate of the RNSP with $0<t\leq 1$, and then obtain another parallel performance guarantee result for (1.5). Finally, conclusion and future work are given in Section 5.

2 Notations and preliminaries

2.1 Notations

Without loss of generality we assume that $n_{1}\leq n_{2}$. For any positive integer $k$, we denote $[k]=\{1,2,\cdots,k\}$, and for any $\Omega\subset[n_{1}]$, we denote $\Omega^{c}=[n_{1}]\setminus\Omega$. We denote the singular value decomposition (SVD) of $H\in\mathbb{R}^{n_{1}\times n_{2}}$ as

[TABLE]

where $\sigma_{i}(H)$ is the $i$th largest singular value of $H$, and $\bm{a}_{H}^{(i)}$ and $\bm{c}_{H}^{(i)}$ are the left and right singular value vectors of $H$, respectively. If there is no confusion caused we will write $\sigma_{i}(H)$, $\bm{a}_{H}^{(i)}$ and $\bm{c}_{H}^{(i)}$ as $\sigma_{i}$, $\bm{a}^{(i)}$ and $\bm{c}^{(i)}$ for simplicity, respectively. For convenience, we denote $H^{(i)}=\sigma_{i}\bm{a}^{(i)}\left(\bm{c}^{(i)}\right)^{T}$, $H_{\Omega}=\sum_{i\in\Omega}H^{(i)}$, and also denote by $\sigma_{\Omega}$ the vector whose element is equal to $\sigma_{i}$ for $i\in\Omega$ and 0 otherwise. Then clearly $H_{[k]}=\sum_{i=1}^{k}H^{(i)}$ and $\|\sigma_{\Omega}\|_{1}=\|H_{\Omega}\|_{*}$. In the end, for any given positive number $\alpha$, we denote $T(\alpha,k)\subset\mathbb{R}^{n_{1}}$ as

[TABLE]

and for any $\bm{y}\in\mathbb{R}^{n_{1}}$, we further denote $U(\alpha,k,\bm{y})\subset\mathbb{R}^{n_{1}}$ as

[TABLE]

where $\|\bm{x}\|_{0}$ is denoted as the number of the nonzero elements in $\bm{x}$.

2.2 Three key lemmas

Before presenting our main results, we need some auxiliary lemmas. We start with introducing the first one, which provides a powerful tool to represent a non-sparse vector by the sparse ones. This lemma was first established by Cai and Zhang in [10], and later was extended by Zhang and Li in [12].

Lemma 2.1**.**

Suppose that $\alpha$ is a positive number and $k$ is a positive integer with $k<n_{1}$. Then $\bm{v}\in\mathbb{R}^{n_{1}}$ obeys $\bm{v}\in T(\alpha,k)$ if and only if $\bm{v}$ is in the convex hull of $U(\alpha,k,\bm{v})$. In particular, any $\bm{v}\in T(\alpha,k)$ can be expressed as

[TABLE]

where $\bm{z}^{(l)}\in U(\alpha,k,\bm{v})$, $0\leq\gamma_{l}\leq 1$ and $\sum_{l}\gamma_{l}=1$. Moreover,

[TABLE]

We also need the following Lemma 2.2, which provides a family of RIC-based conditions under which the RNSP can be guaranteed. More importantly, under these conditions, we will show in Theorem 3.1 that the RNNM model (1.5) is able to robustly recover any matrix that is not necessary to be exactly low-rank.

Lemma 2.2**.**

For any fixed $t>1$ and any positive integer $k<n_{1}$ with $tk<n_{1}$, if the map $\mathcal{A}$ obeys the RIP of order $tk$ with

[TABLE]

then $\mathcal{A}$ have the RNSP with $\beta_{1}>0$ and $0<\beta_{2}<1$. Specifically, for any matrix $H\in\mathbb{R}^{n_{1}\times n_{2}}$ and $\Omega\subset[n_{1}]$ with $|\Omega|=k$, it holds that

[TABLE]

where

[TABLE]

The RNSP, including the classical NSP as a special case, has been demonstrated to be a powerful theoretical tool in providing the robust recovery guarantees of sparse signals or low-rank matrices via some certain constrained optimization problems, see, e.g., [6, 19, 20, 21]. However, so far it is still an open problem to verify whether a given matrix/map obeys the NSP or not, and also to determine the values of two coefficients (i.e., $\beta_{1}$ and $\beta_{2}$) in RNSP. We note that there exist few researchers who focused on the characterization of these two coefficients with some other theoretical tools (such as the RIC and coherence) that are relatively easy to be checked. To the best of our knowledge, the first $2k$-order RIC based coefficient estimate of RNSP was obtained independently by Shen, et al. in [22, Lemma 1] and Foucart in [19, Theorem 5] to fit the sparse recovery scenarios. Later, by using the $tk$-order RIC tool with $t>1$, Ge, et al. in [23, Lemma 2] extended their results to a more general case. Recently, the coherence-based coefficient estimate of RNSP was obtained by Wang, et al. in [24, Lemma 3] to deal with the robust signal recovery from the basis pursuit de-noising [25]. In fact, our Lemma 2.2 can be viewed as an extension of [23, Lemma 2] established for the measurement matrix to that for the measurement map.

Proof of Lemma 2.2.

The proof mainly follows from [23]. When $tk$ is not an integer, let $t^{\prime}=\lceil tk\rceil/k$, then $t^{\prime}>t$ and $t^{\prime}k$ is an integer. In view of this, we here only need to prove Lemma 2.2 when $tk$ is a positive integer for a given $t>1$. Let’s denote the SVD of $H$ as $H=\sum_{i=1}^{n_{1}}\sigma_{i}\bm{a}^{(i)}\left(\bm{c}^{(i)}\right)^{T}$, and

[TABLE]

Then clearly $\Lambda_{1}\cup\Lambda_{2}=\Omega^{c}$ and $\Lambda_{1}\cap\Lambda_{2}=\emptyset$. In what follows, we start with proving that

[TABLE]

To do so, we first show that $|\Lambda_{1}|<(t-1)k$. In fact it holds naturally if $\Lambda_{1}=\emptyset$. When $\Lambda_{1}\neq\emptyset$, we know that

[TABLE]

This also yields the desired result. On the other hand, we can easily induce from the definition of $\Lambda_{1}$ and $\Lambda_{2}$ that

[TABLE]

which, together with Lemma 2.1, indicates that we can express $\sigma_{\Lambda_{2}}$ as

[TABLE]

with $\bm{z}^{(l)}$ satisfying

[TABLE]

By further defining

[TABLE]

where $Z^{(l)}=\sum_{i=1}^{n_{1}}\left(\bm{z}^{(l)}\right)_{i}\bm{a}^{(i)}\left(\bm{c}^{(i)}\right)^{T}$, we can easily induce that both $B^{(l)}$ and $D^{(l)}$ are all rank-$tk$, and $H_{\Lambda_{2}}=\sum_{l}\gamma_{l}Z^{(l)}$. Next, we consider estimating the upper and lower bounds of

[TABLE]

As to the upper bound of $\rho$, we have

[TABLE]

where we have applied the $tk$-order RIP in the last inequality. As to the lower bound of $\rho$, by applying the $tk$-order RIP on $\rho$, we get

[TABLE]

where we have used $\langle\sigma_{\Omega\cup\Lambda_{1}},\bm{z}^{(l)}\rangle=0$ in the first equality and (2.4) in the last inequality. Therefore, combing (2.2) and (2.2) gives

[TABLE]

Therefore,

[TABLE]

where we have used $\sqrt{x^{2}+y^{2}}\leq|x|+|y|$ for any $x,y\in\mathbb{R}$ in the last inequality. This, together with $\|H_{\Omega}\|_{F}\leq\|H_{\Omega\cup\Lambda_{1}}\|_{F}$, directly yields (2.2). The obtained condition (2.1) follows trivially from (2.2) by enforcing $\beta_{2}=\delta_{tk}/\sqrt{(1-(\delta_{tk})^{2})(t-1)}<1$. ∎

In the end, we introduce the last lemma (i.e., Lemma 2.2), which characterizes the relationship between the original solution $X$ and the optimal solution $X^{\sharp}$ of (1.5).

Lemma 2.3**.**

Assume that $X^{\sharp}$ is the solution of (1.5) and $H=X^{\sharp}-X$. If the noisy measurements $\bm{b}=\mathcal{A}(X)+\bm{n}$ are observed with the noise level $\|\bm{n}\|_{2}\leq\epsilon$, then for any subset $\Omega\subset[n_{1}]$ with $|\Omega|=k$, we have

[TABLE]

and

[TABLE]

Proof of Lemma 2.3.

Since $X^{\sharp}$ is the optimal solution of (1.5), we have

[TABLE]

which is equivalent to

[TABLE]

As to the left-hand side (LHS) of (2.9), we have

[TABLE]

As to the right-hand side (RHS) of (2.9), we know

[TABLE]

where we have used [26, Theorem 1] in the first inequality. Then combing (2.9), (2.10), and (2.2) leads to the desired result (2.7), and (2.8) follows trivially from (2.7). ∎

3 Performance guarantee of RNMM model under $tk$-order RIC with $t>1$

With previous preparations in mind, we present our first theoretical result.

Theorem 3.1**.**

For any observed vector $\bm{b}=\mathcal{A}(X)+\bm{n}$ with a bounded constraint $\|\bm{n}\|_{2}\leq\epsilon$, if the $tk$-order RIC of $\mathcal{A}$ with $t>1$ satisfies condition (2.1), then we have

[TABLE]

where $X^{\sharp}$ is the optimal solution of (1.5), and

[TABLE]

Remark 3.2**.**

The condition (2.1) has been obtained previously by Cai and Zhang in [10] for exact/robust signal recovery from (1.2), and it has been proved to be sharp for the exactly rank-$k$ matrix recovery when $t>4/3$. To the best of our knowledge, we first extend nontrivially this condition from the constrained NNM model (1.2) to its unconstrained counterpart, i.e., the unconstrained RNNM model (1.5). On the other hand, note that the obtained coefficients $C_{i}(\beta_{1},\beta_{2})$ (for $i=1,2,3,4$) might seem a bit complicated since they not only involve $\beta_{1},\beta_{2}$, but also involve $k$, $\lambda$ and $\epsilon$. To remedy this problem, we need to do some simplification. We here only take $C_{3}(\beta_{1},\beta_{2})$ and $C_{4}(\beta_{1},\beta_{2})$ for examples. Since

[TABLE]

and

[TABLE]

we thus can induce from (3.2) that

[TABLE]

where $\widehat{C}_{\lambda/\epsilon}$ and $\widetilde{C}_{\lambda/\epsilon}$ are two constants only relying on the map $\mathcal{A}$ and the value of $\lambda/\epsilon$, and they are given as below.

[TABLE]

Note that the induced upper-bound estimate (3.3) also coincide with the ones established in [22, 23, 24, 27] in form.

Remark 3.3**.**

Theorem 3.1 states that if the measurement map $\mathcal{A}$ obeys a certain $tk$-order RIP condition related to $t>1$, any matrix that is not necessary to be exactly low-rank can be robustly recovered from (1.5) for any fixed parameter $\lambda>0$. According to the obtained results, it is difficult to determine a “good” parameter $\lambda$ to yield a “good” solution in general case. In fact, so far it is still an open problem to theoretically determine a general parameter $\lambda$ to make sure that the unconstrained RNNM model (1.5) can perform well. However, if taking a close look at the obtained (3.2) and (3.3), one will find that the selected parameter $\lambda$ should not be much too large or small. Furthermore, if the desired matrix $X$ is assumed to be exactly rank-$k$, then we can induce from (3.2) that

[TABLE]

Obviously, if we desire a optimal solution with recovery error as small as possible from (1.5), we need to make sure that the value of $C_{4}(\beta_{1},\beta_{2})$ is also as small as possible with respect to the parameter $\lambda$. Considering that

[TABLE]

where the equality in (a) holds when $\lambda$ satisfies

[TABLE]

a ideal selection of parameter $\lambda$ is to set it as in (3.4), which is related to the two coefficient estimates of RNSP of the map $\mathcal{A}$, noise level $\epsilon$ and also the rank parameter $k$. In realistic situations, such a setting of $\lambda$ is impractical. However, from (3.4) we can capture some information to help set a proper $\lambda$, i.e., the value of $\lambda$ is proportional to that of $\epsilon^{2}$, and inversely proportional to that of $k$.

Now, we present the proof of Theorem 3.1 as follows.

Proof.

We start with proving (3.1). Let’s define $\widehat{\Omega}=[k]$ and $H=X^{\sharp}-X$. Then by using Lemma 2.2 and Lemma 2.3 with $\Omega=\widehat{\Omega}$, we have

[TABLE]

Due to (2.1), $\beta_{2}<1$ and thus we induce from (3) that

[TABLE]

This directly leads to

[TABLE]

which is the desired (3.1).

Before proving (3.2), let’s define $\Omega_{1}=\{k+1,k+2,\cdots,2k\}$, $\Omega_{2}=\{2k+1,2k+2,\cdots,3k\}$, $\Omega_{3}=\{3k+1,3k+2,\cdots,4k\}$, and so on. Thus for $i=2,3,4,\cdots$, we have $\|H_{\Omega_{i}}\|_{F}\leq\|H_{\Omega_{1}}\|_{F}$, and therefore,

[TABLE]

where the last inequality is due the fact that

[TABLE]

Note that by combining Lemma 2.2 and Lemma 2.3 with $\Omega=\widehat{\Omega}$ again, we can provide two upper-bound estimates of $\|H_{\widehat{\Omega}}\|_{F}$ and $\|H_{\widehat{\Omega}^{c}}\|_{*}$, respectively, which are independent from each other. First, as to that of $\|H_{\widehat{\Omega}}\|_{F}$, we have

[TABLE]

which is equivalent to

[TABLE]

Similarly, we can also easily get the upper-bound estimate of $\|H_{\widehat{\Omega}^{c}}\|_{*}$ as below.

[TABLE]

On the other hand, by combining (3) and Lemma 2.2 with $\Omega=\Omega_{1}$, we have

[TABLE]

which, together with (3) and (3.8), yields

[TABLE]

Now by combining (3), (3), (3.8) and (3), we can estimate $\|H\|_{F}$ as follows.

[TABLE]

which, together with (3.1), yields

[TABLE]

This completes the proof of (3.2). ∎

4 $tk$-order RIC based coefficient estimate of RNSP with $0<t\leq 1$

In the previous section, a family of $tk$-order RIP conditions and their resultant recovery error estimate results are established for the robust matrix recovery from the unconstrained RNNM model (1.5). As is seen from Theorem 3.1 and its proof, Lemma 2.2, i.e., the RNSP with $tk$-order RIC based coefficient estimate, plays a vital role in establishing the desired results. Unfortunately, Lemma 2.2, as well as its resultant Theorem 3.1, only considers the case of $t>1$. In this section, we will show that under the $tk$-order RIP condition with $0<t\leq 1$, (1.5) is still able to provide a robust recovery performance. Before moving on, we have to introduce [11, Lemma 1] since it will be frequently used in the proof of our main results, and one can find it in Lemma 4.1.

Lemma 4.1**.**

Let $\bm{w}\in\mathbb{R}^{k}$ be a vector with $\bm{w}=[w_{1},w_{2},\cdots,w_{k}]^{T}$. Choose all subsets $S_{i}\subseteq[k]$ with $|S_{i}|=s<k$, $i\in I$ with $|I|=\binom{k}{s}$, then we have

[TABLE]

and

[TABLE]

Now we are ready to present our second theoretical result, i.e., the $tk$-order RIC based coefficient estimate of RNSP with $0<t\leq 1$.

Theorem 4.2**.**

For any fixed $0<t\leq 1$ and any positive integer $k<n_{1}$ with $tk<n_{1}$, if the map $\mathcal{A}$ obeys the RIP of order $tk$ with

[TABLE]

where $\theta_{1}=\frac{1}{t}\sqrt{\frac{2-t}{1-t}}$, then $\mathcal{A}$ obeys the RNSP with $\widehat{\beta}_{1}>0$ and $0<\widehat{\beta}_{2}<1$. Specifically, for any matrix $H\in\mathbb{R}^{n_{1}\times n_{2}}$ and any subset $\Omega\subset[n_{1}]$ with $|\Omega|=k$, it holds that

[TABLE]

where

[TABLE]

with $\theta_{2}=2/t$, and $\psi(\cdot)$ and $\varphi(\cdot)$ being given in (4) and (4), respectively.

Remark 4.3**.**

To the best of our knowledge, Theorem 4.2 for the first time presents the $tk$-order RIC based coefficient estimate of the RNSP in the case of $0<t\leq 1$. This theorem, together with Lemma 2.2, affirmatively answers under what kind of $tk$-order RIP condition with $t>0$, RNSP will hold. Note that we can also resort to Theorem 4.2 to yield a similar result with Theorem 3.1, and one can find it in Theorem 4.4. Since the proof Theorem 4.4 is almost same with that of Theorem 3.1, we here omit it.

Theorem 4.4**.**

For any observed vector $\bm{b}=\mathcal{A}(X)+\bm{n}$ with a bounded constraint $\|\bm{n}\|_{2}\leq\epsilon$, if the $tk$-order RIC of $\mathcal{A}$ satisfies (4.1), then we have

[TABLE]

where $X^{\sharp}$ is denoted as the optimal solution of (1.5).

Remark 4.5**.**

Theorem 4.4 and Theorem 3.1 indicate that under a certain $tk$-order RIP condition with $t>0$, the unconstrained RNNM model (1.5) is able to provide a robust recovery performance for any matrix that is not necessary to be exactly low-rank. Note that the previous analysis results on Theorem 3.1 still apply to Theorem 4.4 if one replaces $\beta_{1}$ and $\beta_{2}$ with $\widehat{\beta}_{1}$ and $\widehat{\beta}_{2}$, respectively. On the other hand, one may wonder how the obtained condition (4.1) performs when compared with the sharp condition (1.4) established for the constrained NNM model (1.2).

Figure 1 plots the comparison between these two recovery conditions. Unfortunately, our condition (4.1) is a bit weaker than the sharp condition (1.4). Considering the fact that problems of type (1.5) and type (1.2) are not completely equivalent in both theoretical and applied aspects, it is still an open problem to determine whether the sharp condition (1.4) for $0<t\leq 4/3$ is appropriate for (1.5) or not. According to the established Lemma 2.2 and Theorem 3.1, it is expected that the condition (4.1) with $0<t\leq 1$ for both Theorem 4.2 and Theorem 4.4 can be further improved to the sharp condition (1.4) with $0<t\leq 4/3$. We hope we can solve this problem in the future.

Proof of Theorem 4.2.

Our proof is inspired by [11]. We here only prove the case when $tk$ is an integer since the case when $tk$ is not an integer can be induced easily. Note that for any subset $\Omega\subset[n_{1}]$ with $|\Omega|=k$ and a fixed subset $\widehat{\Omega}=[k]$, $\|H_{\Omega}\|_{F}\leq\|H_{\widehat{\Omega}}\|_{F}$ and $\|H_{\widehat{\Omega}^{c}}\|_{*}\leq\|H_{\Omega^{c}}\|_{*}$ always hold, and hence we will prove

[TABLE]

to complete the proof of (4.2). We start with denoting the SVD of $H$ as $H=\sum_{i=1}^{n_{1}}\sigma_{i}\bm{a}^{(i)}\left(\bm{c}^{(i)}\right)^{T}$ and $H^{(i)}=\sigma_{i}\bm{a}^{(i)}\left(\bm{c}^{(i)}\right)^{T}$. We denote $a=b=tk/2$ when $tk$ is an even number, and $a=(tk+1)/2$ and $b=(tk-1)/2$ when $tk$ is an odd number. We also denote all the possible index $\Delta_{i},\Gamma_{j}\subseteq\widehat{\Omega}$ with $|\Delta_{i}|=a$ and $|\Gamma_{j}|=b$, respectively, where $i\in I(|I|=\binom{k}{a})$ and $j\in J(|J|=\binom{k}{b})$. According to Lemma 4.1, we directly have

[TABLE]

Besides, we need to introduce the following partition for $\widehat{\Omega}^{c}$, i.e.,

[TABLE]

By using the similar manipulations as in proof of Lemma 2.2, we can express $\sigma_{\Lambda_{4}}$ and $\sigma_{\Lambda_{6}}$ as

[TABLE]

respectively, with $\bm{u}^{(l)}$ and $\bm{v}^{(l)}$ satisfying

[TABLE]

Let’s denote $E^{(l)}=H_{\Lambda_{3}}+U^{(l)}$ and $G^{(l)}=H_{\Lambda_{3}}+U^{(l)}$, where

[TABLE]

then we have $H_{\Lambda_{4}}=\sum_{l}\eta_{l}U^{(l)}$ and $H_{\Lambda_{6}}=\sum_{l}\widetilde{\eta}_{l}V^{(l)}$. We also need to denote

[TABLE]

where $\theta\geq 1$ is a number which will be determined later. Since $H_{\Delta_{i}}$, $E^{(l)}$, $H_{\Gamma_{j}}$, $G^{(l)}$ are rank-$a$, -$b$, -$b$, and -$a$, respectively, we can easily induce that any linear combination of $H_{\Delta_{i}}$ and $E^{(l)}$, as well as that of $H_{\Gamma_{j}}$ and $G^{(l)}$, is rank-$tk$. Let’s first apply $tk$-order RIP on $\kappa_{a,b}$. This directly gives

[TABLE]

where we have used (4.4) and (4.7) in the last equality. Similarly, we can also apply $tk$-order RIP on $\widetilde{\kappa}_{a,b}$ to get

[TABLE]

Therefore, combing (4) and (4) yields

[TABLE]

Due to the fact that

[TABLE]

we can further induce from (4) that

[TABLE]

On the other hand, according to the definition of $\kappa_{a,b}$ and $\widetilde{\kappa}_{a,b}$ we can induce that

[TABLE]

To simplify $\tau$ and $M$, from the definition of $\Delta_{i}$ and the SVD of $H$ we can induce that

[TABLE]

and

[TABLE]

where we have used Lemma (4.1) again. Similarly, we can also get

[TABLE]

and $\sum_{j\in J}H_{\Gamma_{j}}=\binom{k-1}{b-1}H_{\widehat{\Omega}}$. Those indicate that

[TABLE]

and

[TABLE]

and thus we can equivalent write (4) as

[TABLE]

where $c_{\theta}=[(a-b)^{2}-2\theta(2-t)ab]$. Since $\theta\geq 1$, $0<t\leq 1$, and $a,b$ have been well defined, we can easily know that $c_{\theta}<0$. Note that there are exactly $\binom{k-a}{b}$ sets $\Gamma_{j}$ for a fixed $\Delta_{i}$, exactly $\binom{k-b}{a}$ sets $\Delta_{i}$ for a fixed $\Gamma_{i}$, and exactly $\binom{k-2}{a+b-2}$ sets with $\Delta_{i}\cap\Gamma_{j}=\emptyset$ for two fixed indices $p,q$ with $p\neq q$ and $p,q\in\Delta_{i}\cup\Gamma_{j}$. Therefore, by means of Lemma 4.1 and some simple calculation, we get

[TABLE]

Moreover, applying $tk$-order RIP on the left-hand side (LHS) of (4) and also using (4.4) again, we also have

[TABLE]

Now combining (4) and (4) yields

[TABLE]

As to the LHS of (4), we can directly induce from (4) that

[TABLE]

where we have used $c_{\theta}<0$. As to the right-hand side (RHS) of (4), we induce from (4) that

[TABLE]

where, with the aid of [8, Lemma 4.1], we have used

[TABLE]

Therefore, we can induce from (4.18) and (4) that

[TABLE]

which is also equivalent to

[TABLE]

where $f(\theta)=(2-t)\{t\theta-[1+(2-t)\theta]\delta_{tk}\}-\theta^{2}(1-t)t^{2}\delta_{tk}-\theta^{2}(3t-2)ab\delta_{tk}/k^{2}$.

Case 1: $0<t\leq 2/3$. In this case, we know from $f(\theta)$ that

[TABLE]

To make sure $f(\theta)>0$, we only need to set $\psi(\theta)>0$, i.e.,

[TABLE]

To obtain as large an upper bound as possible, we need to set $\theta=\theta_{1}=\frac{1}{t}\sqrt{\frac{2-t}{1-t}}$, and thus the largest upper bound with respect to $\theta$ will take the form

[TABLE]

One can easily check that $\theta_{1}\geq 1$ holds for any $0<t\leq 2/3$. Based on the above settings, we can further know from (4.20) that

[TABLE]

and hence get

[TABLE]

which is the desired (4.2) for $0<t\leq 2/3$. To guarantee that (4.2) satisfies the RNSP with $\widehat{\beta}_{1}>0$ and $0<\widehat{\beta}_{2}<1$, we have to set $\widehat{\beta}_{2}=\theta_{1}\sqrt{(2-t)\delta_{tk}/\psi(\theta_{1})}<1$, i.e.,

[TABLE]

Obviously, the obtained condition (4.23) is the desired condition (4.1) for $0<t\leq 2/3$, which is also included in (4.22).

Case 2: $2/3<t\leq 1$. In this case, by using $ab\leq t^{2}k^{2}/4$, we can also know from $f(\theta)$ that

[TABLE]

To make sure that $f(\theta)>0$ and $\delta_{tk}$ has as large an upper bound as possible, By using the similar manipulations as in Case 1, we select $\theta=\theta_{2}=2/t$, and thus get $\delta_{tk}<t/2$. Obviously, $\theta_{2}\geq 1$ holds for any $2/3<t\leq 1$. Furthermore, we can also know from (4.20) that

[TABLE]

and hence get

[TABLE]

which is the desired (4.2) for $2/3<t\leq 1$. Similarly, to enforce (4.2) to obey the RNSP with $\widehat{\beta}_{1}>0$ and $0<\widehat{\beta}_{2}<1$, we have to set $\widehat{\beta}_{2}=\theta_{2}\sqrt{\delta_{tk}/\varphi(\theta_{2})}<1$, i.e.,

[TABLE]

which is the desired condition (4.1) for $2/3<t\leq 1$. Combining Case 1 and Case 2, we obtain the desired (4.3), and thus establish the results showed in Theorem 4.2.

∎

5 Conclusion and future work

This paper has considered the robust matrix recovery from the unconstrained RNMM model. First, equipped with the powerful $tk$-order RIP tool for $t>0$, we developed a family of $tk$-order RIC based coefficient estimates for the RNSP. To the best of our knowledge, the obtained RNSP results in the case of $0<t\leq 1$ have not been explored before. Furthermore, by mean of these RNSP results, some upper-bound estimates of error were established for the unconstrained RNMM model to guarantee the robust matrix recovery. As we have pointed out in Remark 4.5, one of our future work will focus on extending the condition (4.1) with $0<t\leq 1$ to the sharp condition (1.4) with $0<t\leq 4/3$. Besides, determining a proper from the theoretical aspect for the unconstrained RNMM model will be another future work.

Acknowledgement

The authors would like to thank the editors and referees for their valuable comments that improve the presentation of this paper.