Deterministic Analysis of Weighted BPDN With Partially Known Support Information

TL;DR

This paper provides a deterministic analysis of weighted BPDN for signal recovery with partial support information, establishing conditions based on RIC and deriving error bounds, extending previous stochastic results.

Contribution

It introduces a deterministic framework for analyzing weighted BPDN with PKSI, extending existing conditions and error estimates from constrained to unconstrained models.

Findings

Established sufficient RIC-based conditions for robust recovery

Extended analysis from constrained to unconstrained weighted BPDN

Derived comparable error estimates for signal recovery

Abstract

In this paper, with the aid of the powerful Restricted Isometry Constant (RIC), a deterministic (or say non-stochastic) analysis, which includes a series of sufficient conditions (related to the RIC order) and their resultant error estimates, is established for the weighted Basis Pursuit De-Noising (BPDN) to guarantee the robust signal recovery when Partially Known Support Information (PKSI) of the signal is available. Specifically, the obtained conditions extend nontrivially the ones induced recently for the traditional constrained weighted $\ell_{1}$-minimization model to those for its unconstrained counterpart, i.e., the weighted BPDN. The obtained error estimates are also comparable to the analogous ones induced previously for the robust recovery of the signals with PKSI from some constrained models. Moreover, these results to some degree may well complement the recent investigation of the weighted BPDN which is based on the stochastic analysis.

Equations

$$\mathbf{b}=A\hat{\mathbf{x}}+\mathbf{z},$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^n}{\min }\Vert \mathbf{x}{\Vert }_{1}s.t.\Vert \mathbf{b}-A\mathbf{x}{\Vert }_2\le ϵ,$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^n}{\min }\Vert \mathbf{x}{\Vert }_{\mathbf{w},1}\triangleq \sum _{i=1}^n{\mathbf{w}}_i|{\mathbf{x}}_i|s.t.\Vert \mathbf{b}-A\mathbf{x}{\Vert }_2\le ϵ,$$

$$\displaystyle\begin{split}\bm{w}_{i}=\left\{\begin{array}[]{ll}w\in[0,1],&i\in K\\ 1,&i\in K^{c}\end{array}\right.,\end{split}$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^n}{\min }\Vert \mathbf{x}{\Vert }_{\mathbf{w},1}+\frac{1}{2\lambda }\Vert \mathbf{b}-A\mathbf{x}{\Vert }_2^2,$$

$${\mathbf{x}}_{\max (s)}={\mathrm{argmin}}_{\Vert \mathbf{y}{\Vert }_0\le s}\Vert \mathbf{y}-\mathbf{x}{\Vert }_2.$$

$$(1-\delta )\Vert \mathbf{x}{\Vert }_2^2\le \Vert A\mathbf{x}{\Vert }_2^2\le (1+\delta )\Vert \mathbf{x}{\Vert }_2^2$$

$$\displaystyle\begin{split}d=\left\{\begin{array}[]{ll}1-\alpha\rho+\max\{\alpha,1-\alpha\}\rho,&0\leq w<1\\ 1,&w=1\end{array}\right.\end{split}.$$

$$\Vert A\mathbf{h}{\Vert }_2^2-2ϵ\Vert A\mathbf{h}{\Vert }_2\le 4\lambda \left(w\Vert {\hat{\mathbf{x}}}_{E^c}{\Vert }_1+(1-w)\Vert {\hat{\mathbf{x}}}_{K^c\cap E^c}{\Vert }_1\right)+2\theta \sqrt{k}\lambda \Vert {\mathbf{h}}_{\max (dk)}{\Vert }_2-2\lambda \Vert {\mathbf{h}}_{E^c}{\Vert }_1$$

$$\Vert {\mathbf{h}}_{E^c}{\Vert }_1\le 2(w\Vert {\hat{\mathbf{x}}}_{E^c}{\Vert }_1+(1-w)\Vert {\hat{\mathbf{x}}}_{K^c\cap E^c}{\Vert }_1)+\theta \sqrt{k}\Vert {\mathbf{h}}_{\max (dk)}{\Vert }_2+\frac{ϵ}{\lambda }\Vert A\mathbf{h}{\Vert }_2,$$

$$\Vert {\mathbf{x}}^{♯}{\Vert }_{\mathbf{w},1}+\frac{1}{2\lambda }\Vert \mathbf{b}-A{\mathbf{x}}^{♯}{\Vert }_2^2\le \Vert \hat{\mathbf{x}}{\Vert }_{\mathbf{w},1}+\frac{1}{2\lambda }\Vert \mathbf{b}-A\hat{\mathbf{x}}{\Vert }_2^2,$$

$$\Vert A\mathbf{h}{\Vert }_2^2-2⟨\mathbf{z},A\mathbf{h}⟩\le$$

$$\text{LHS}\ge \Vert A\mathbf{h}{\Vert }_2^2-2ϵ\Vert A\mathbf{h}{\Vert }_2.$$

$$\frac{\text{RHS}}{2\lambda }\le$$

$$w\Vert {\mathbf{h}}_E{\Vert }_1+(1-w)\Vert {\mathbf{h}}_U{\Vert }_1\le w\sqrt{k}\Vert {\mathbf{h}}_E{\Vert }_2+(1-w)\sqrt{(1+\rho -2\alpha \rho )k}\Vert {\mathbf{h}}_U{\Vert }_2\le \theta \sqrt{k}\Vert {\mathbf{h}}_{\max (dk)}{\Vert }_2,$$

$$\theta =w+(1-w)\sqrt{1+\rho -2\alpha \rho }.$$

$$\frac{\text{RHS}}{2\lambda }\le 2\left(w\Vert {\hat{\mathbf{x}}}_{E^c}{\Vert }_1+(1-w)\Vert {\hat{\mathbf{x}}}_{K^c\cap E^c}{\Vert }_1\right)+\theta \sqrt{k}\Vert {\mathbf{h}}_{\max (dk)}{\Vert }_2-\Vert {\mathbf{h}}_{E^c}{\Vert }_1.$$

$$\Vert {\mathbf{h}}_S{\Vert }_2\le {\beta }_1\Vert A\mathbf{h}{\Vert }_2+\frac{{\beta }_2}{\sqrt{(t-g)k}}\Vert {\mathbf{h}}_{S^c}{\Vert }_1,$$

$${\beta }_1=\frac{2}{(1-{\delta }_{tk})\sqrt{1+{\delta }_{tk}}}\text{ and }{\beta }_2=\frac{{\delta }_{tk}}{\sqrt{1-({\delta }_{tk}{)}^2}}.$$

$${\beta }_1<$$

$${\beta }_1>$$

$$\frac{1}{\sqrt{t-d}-\theta {\beta }_2}<\frac{{\theta }^{-1}}{\sqrt{(t-d)/(t-d+{\theta }^2)}-{\delta }_{tk}}.$$

$$S_1=$$

$$S_2=$$

$$\Vert {\mathbf{h}}_{S\cup S_1}{\Vert }_2\le {\beta }_1\Vert A\mathbf{h}{\Vert }_2+\frac{{\beta }_2}{\sqrt{t-g}}\Vert {\mathbf{h}}_{S^c}{\Vert }_1.$$

$${\delta }_{tk}<\sqrt{\frac{t-d}{t-d+{\theta }^2}},$$

$$\Vert {\mathbf{x}}^{♯}-\hat{\mathbf{x}}{\Vert }_2\le C_1({\beta }_1,{\beta }_2)\left(w\Vert {\hat{\mathbf{x}}}_{E^c}{\Vert }_1+(1-w)\Vert {\hat{\mathbf{x}}}_{K^c\cap E^c}{\Vert }_1\right)+C_2({\beta }_1,{\beta }_2),$$

$$C_1({\beta }_1,{\beta }_2)=$$

$$C_2({\beta }_1,{\beta }_2)=$$

$$\Vert {\mathbf{x}}^{♯}-\hat{\mathbf{x}}{\Vert }_2\le {\tilde{C}}_1$$

$${\tilde{C}}_1\triangleq$$

$${\tilde{C}}_2\triangleq$$

Full text

I Introduction

Compressed/compressive Sensing (CS), see, e.g., [1, 2, 3], has captured a lot of attention of the researchers in a wide range of fields over the past decade. In CS, one gets the observations of signal $\widehat{\bm{x}}\in\mathbb{R}^{n}$ via the following model

[TABLE]

where $A\in\mathbb{R}^{m\times n}(m\ll n)$ is called the measurement matrix and $\bm{z}\in\mathbb{R}^{m}$ denotes the additive noise that satisfies a certain constraint. One of the key goals of CS is to effectively recover the original signal $\widehat{\bm{x}}$ based on $A$ and $\bm{b}$. It has been shown that, if $\widehat{\bm{x}}$ is $k$-sparse with $k<m$ and $A$ satisfies certain conditions related to $k$, see, e.g., [4, 5, 6, 7, 8], then one can achieve this goal by solving an $\ell_{1}$-minimizing problem, i.e.,

[TABLE]

where $\epsilon\geq 0$ represents for the noise level, and we take $\epsilon=0$ if there is no noise, i.e., $\bm{z}=0$.

The above $\ell_{1}$-minimization approach has been demonstrated to be effective in robust signal recovery. However, it does not incorporate any prior information on signal support since the $\ell_{1}$-norm treats the entries of variable $\bm{x}$ equally. In fact in many practical applications such as the time-series signal processing, see, e.g., [9, 10, 11], it is often possible to estimate a part of the signal support information. It thus becomes very necessary and important to use such prior information to further enhance the recovery performance of (2). This consideration directly leads to the following weighted $\ell_{1}$-minimization problem

[TABLE]

where $\bm{w}\triangleq[\bm{w}_{1},\cdots,\bm{w}_{n}]^{T}$ denote the weights. For simplicity, in this paper we only consider a binary choice of $\bm{w}$, i.e.,

[TABLE]

where $K\subset[n]\triangleq\{1,2,\cdots,n\}$ is a given set, which models the Partially Known Support Information (PKSI) of $\widehat{\bm{x}}$. This problem has been well investigated in the past few years, see, e.g., [12, 13, 14, 15, 16, 17, 18, 19]. It was proved by Friedlander, et al. in [12] that if $K$ includes half of the accurate support of $\bm{\widehat{x}}$ at least, then (3) will perform robustly under much weaker conditions than the analogous ones for (2). In [15], Flinth studied the optimal choice for general weights. Recently, Chen, et al. in [18] and [19] obtained some much tighter conditions for (3), and these conditions were proved to be sharp when the desired signal $\bm{\widehat{x}}$ is exactly sparse and is also measured without noise.

In this paper, we consider the robust recovery of the signals with PKSI via the weighted Basis Pursuit De-Noising (BPDN)

[TABLE]

where $\lambda$ is a positive parameter. Obviously, (4) will be reduced to the widely known BPDN if one sets $w=1$ (i.e., no support information is available). Although there exists a large amount of research on the BPDN, see, e.g., [20, 21, 22, 23, 24, 25, 26, 27], the theoretical analysis of (4) for sparse recovery is relatively less studied. We note that Lian, et al. recently studied (4) from both theoretical and experimental aspects in [28], where they called it weighted LASSO. However, their obtained results are established on the stochastic strategy, and they are totally different from ours that are established in a deterministic manner.

The main contribution of this paper is that a series of (tight) sufficient conditions as well as their resultant error estimates are established for (4) with the help of the Restricted Isometry Property (RIP) [1], which to some degree well complement the recent theoretical analysis of the weighted BPDN (see, [28]) that is based on the stochastic strategy.

II Notations and Preliminaries

In this section, we first introduce some basic notations. For any given index set $S\subset[n]$, we denote $\bm{h}_{S}$ as a vector whose entries $(\bm{h}_{S})_{i}=\bm{h}_{i}$ for $i\in S$ and 0 otherwise, and also denote the best $s$-term approximate $\bm{x}_{\max(s)}$ of any signal $\bm{x}\in\mathbb{R}^{n}$ as

[TABLE]

Definition 1**.**

A matrix $A\in\mathbb{R}^{m\times n}$ is said to obey the RIP of order $k$, if there exists a constant $\delta\in(0,1)$ such

[TABLE]

for every $k$-sparse signal $\bm{x}\in\mathbb{R}^{n}$. The smallest positive $\delta$ that satisfies (5) is denoted by $\delta_{k}$ 111When $k$ is not an integer, we define $\delta_{k}$ as $\delta_{\lceil k\rceil}$. and is known as the Restricted Isometry Constant (RIC).

We also need the following two lemmas.

Lemma 1**.**

Assume that $E,K\subseteq[n]$ are two sets with $|E|=k$, $|K|=\rho k$ and $|E\cap K|=\alpha\rho k$ for some $\rho\geq 0$ and $0\leq\alpha\leq 1$, and define

[TABLE]

If $\bm{b}$ is observed through (1) with the noise constrain $\|\bm{z}\|_{2}\leq\epsilon$, then for the optimal solution $\bm{x}^{\sharp}$ of (4), we have

[TABLE]

and

[TABLE]

where $\bm{h}=\bm{x}^{\sharp}-\bm{\widehat{x}}$ and $\theta$ is denoted by (12).

Proof:

Since $\bm{x}^{\sharp}$ is the optimal solution of (4), we have

[TABLE]

which is equivalent to

[TABLE]

As to the left-hand side of (9), we have

[TABLE]

As to the right-hand side of (9), we know from [12] that

[TABLE]

where $U=(K^{c}\cap E)\cup(K\cap E^{c})$. Since $d\geq 1$ and $|U|=(1+\rho-2\alpha\rho)k\leq dk$, then $\|\bm{h}_{E}\|_{2}\leq\|\bm{h}_{\max(dk)}\|_{2}$ and $\|\bm{h}_{U}\|_{2}\leq\|\bm{h}_{\max(dk)}\|_{2}$, and thus clearly

[TABLE]

where

[TABLE]

This directly turns (11) to be the following inequality

[TABLE]

Therefore, combing (10) and (13) leads to the desired (7), and (8) follows trivially from (7). ∎

Lemma 2**.**

For any $g\geq 1$ if $A\in\mathbb{R}^{m\times n}$ satisfies the RIP of order $tk$ with RIC $\delta_{tk}$ and $t>g$, then for any vector $\bm{h}\in\mathbb{R}^{n}$ and any subset $S\subset[n]$ with $|S|=gk$, it holds that

[TABLE]

where

[TABLE]

Remark 1**.**

*It is easy to know from Lemma 2 that both $\beta_{1}$ and $\beta_{2}$ are two monotone increasing functions on the variable $\delta_{tk}$. Therefore if one restricts $\delta_{tk}$ to (19), it will be clear that *

[TABLE]

and

[TABLE]

Proof:

The proof mainly follows from that of [25, Lemma 2]. We here only give some key steps.

Step 1: For a given $t>g$, we start with denoting

[TABLE]

Step 2: Using the similar skills in [25], one can prove

[TABLE]

Step 3: Proving (14) by (18) and $\|\bm{h}_{S}\|_{2}\leq\|\bm{h}_{S\cup S_{1}}\|_{2}$ .

These three steps are sufficient to prove Lemma 2 when $tk$ is an integer. When $tk$ is not an integer, we define $t^{\prime}=\lceil tk\rceil/k$, then $t^{\prime}k$ is an integer and $\delta_{tk}=\delta_{t^{\prime}k}$. Obviously, Lemma 2 still holds in such case. In summary, Lemma 2 will hold no matter whether or not $tk$ is an integer. ∎

III Main Results

With preparations above, we now give the main results.

Theorem 1**.**

Assume that $\bm{b}$ is observed via (1) with $\|\bm{z}\|_{2}\leq\epsilon$ and $E$ is denoted by $E=\text{supp}(\bm{\widehat{x}}_{\max(k)})$. Let $K\subseteq[n]$ be defined as in Lemma 1. If the measurement matrix $A$ satisfies

[TABLE]

where $d$ and $\theta$ are denoted by (6) and (12), respectively, then

[TABLE]

where $\bm{x}^{\sharp}$ is the optimal solution of (4) and

[TABLE]

with $r=\sqrt{t-d}$, $r_{1}=\sqrt{t-1}$ and $f_{i}(\cdot)$ for $i=1,2,3$ being denoted by (32), (33) and (34), respectively.

Remark 2** (Recovery Condition).**

The established condition (19) coincides with the one obtained recently by Chen, et al. in [18], which has been proved to be sharp for the exactly sparse signal recovery under noise-free measurements. However, their goal was to recover the signal with PKSI using the constrained model. On the other hand, our condition (19) in fact is not a simple extension of the one in [18], but is obtained in a totally different way. We refer the interested readers to [18] for more detailed discussion on (19) and its potential corollaries.

Remark 3** (Error Estimate).**

It seems that the obtained error estimate (20) shows a bit complicated since it integrates $\lambda$ and $\epsilon$ together. In what follows, we provide three special cases of (20) by selecting some simple but meaningful $\lambda$’s and/or $\epsilon$’s.

Case 1): Suppose that $\lambda=\epsilon(\neq 0)$, then by using (15), (16) and (17) we can deduce directly from (20) that

[TABLE]

and

[TABLE]

where

[TABLE]

This directly yields

[TABLE]

This new error estimate also coincides with the ones in [24, 25, 26, 27] in form, which are induced for the unconstrained models. However, their results do not take PKSI into consideration.

Case 2): Suppose that $\lambda=\epsilon/\sqrt{k}(\neq 0)$. Similar to the above analysis in Case 1, we can also obtain that

[TABLE]

where

[TABLE]

This result coincides with the ones induced for the traditional constrained models in form, see, e.g., [5, 12, 13], which to some extent indicates theoretically that the unconstrained model (4) and the constrained model (3) are equivalence in robust recovery of any (sparse) signals with PKSI.

Case 3): Suppose that $\epsilon=0$, i.e., $\bm{z}=0$. In such case, it is also easy to deduce from (20) that

[TABLE]

where

[TABLE]

According to the above error estimate, it seems impossible to exactly recover any sparse signals through (4) in the absence of noise. However, if one sets the parameter $\lambda$ to be a sufficient small positive number, the error between $\bm{x}^{\sharp}$ and $\bm{\widehat{x}}$ will tend to depend only on the available PKSI of the original signal itself. On the other hand, from the viewpoint of non-uniform recovery [3], it has been shown that under certain conditions, one can successfully recover some (specific) sparse signals from the BPDN, see, e.g., [20]. This may brings possibility for (4) to realize the exact recovery of some sparse signals with PKSI when certain conditions are satisfied. More discussion on non-uniform recovery is beyond the scope of this paper, and we refer the interested readers to [3] and [29] for details.

Remark 4**.**

Due to the limited space, we can not discuss more on the obtained results in this paper. We refer the interested readers to the supplementary material for more discussion on the weight choice and its resultant performance analysis.

Proof:

We first denote $\bm{h}=\bm{x}^{\sharp}-\bm{\widehat{x}}$, $F=\text{supp}(\bm{h}_{\max(dk)})$, $r=\sqrt{t-d}$, and $\eta=w\|\bm{\widehat{x}}_{E^{c}}\|_{1}+(1-w)\|\bm{\widehat{x}}_{K^{c}\cap E^{c}}\|_{1}$. Then we have

[TABLE]

and also know from Lemma 2 (with $S=F$) that

[TABLE]

Besides, combing (8), (21) and (22) directly yields

[TABLE]

where we used the condition (19) and thus

[TABLE]

for the last inequality. Similarly, we can also deduce from (8), (21) and (22) that

[TABLE]

On the other hand, let $G$ denote the index set of the $k$ largest entries of $\bm{h}_{F^{c}}$ in magnitude. Then we can know from Lemma 2 (with $S=G$) and [24, inequality (2.3)] that

[TABLE]

where $r_{1}=\sqrt{t-1}$. Then using (III) and (25), we have

[TABLE]

where we used $\|\bm{h}_{F}\|_{1}\leq\sqrt{dk}\|\bm{h}_{F}\|_{2}$ in the first inequality.

Now we estimate the upper bound of $\|A\bm{h}\|_{2}$. We first know from (7), (21) and (22) that

[TABLE]

which is equal to

[TABLE]

Using (24) again, we can further deduce from (28) that

[TABLE]

This directly leads to

[TABLE]

Based on (III), we can give two new upper bound estimates for $\|\bm{h}_{F}\|_{2}$ and $\|\bm{h}_{F^{c}}\|_{1}$, respectively, i.e.,

[TABLE]

Now combing (26) and (III), together with (III)-(31), we have

[TABLE]

where

[TABLE]

This completes the proof. ∎

IV Conclusion

This paper aims to provide a deterministic (non-stochastic) analysis for the sparse recovery of signals with partially known support information from the weighted BPDN. Equipped with the powerful RIC notation, we established a series of sufficient conditions and their resultant error estimates. These theoretical results, to some degree, are well complementary for the recent ones of the weighted BPDN established in a stochastic manner.