Column normalization of a random measurement matrix

TL;DR

This paper demonstrates that column normalization of certain random matrices with iid entries does not necessarily ensure good sparse recovery, even with controlled moment growth, challenging assumptions in compressed sensing.

Contribution

It constructs specific random vectors showing that column normalization alone is insufficient for guaranteeing sparse recovery properties.

Findings

Column normalization does not guarantee sparse recovery.

Constructed vectors satisfy moment growth conditions.

Normalized matrices can fail exact reconstruction with high probability.

Abstract

In this note we answer a question of G. Lecu\'{e}, by showing that column normalization of a random matrix with iid entries need not lead to good sparse recovery properties, even if the generating random variable has a reasonable moment growth. Specifically, for every $2 \leq p \leq c_1\log d$ we construct a random vector $X \in R^d$ with iid, mean-zero, variance $1$ coordinates, that satisfies $\sup_{t \in S^{d-1}} \|<X,t>\|_{L_q} \leq c_2\sqrt{q}$ for every $2\leq q \leq p$. We show that if $m \leq c_3\sqrt{p}d^{1/p}$ and $\tilde{\Gamma}:R^d \to R^m$ is the column-normalized matrix generated by $m$ independent copies of $X$, then with probability at least $1-2\exp(-c_4m)$, $\tilde{\Gamma}$ does not satisfy the exact reconstruction property of order $2$.

Equations

$$\min \Vert t{\Vert }_1\mathrm{subject}\mathrm{to}\Gamma v=\Gamma t,$$

$$\|\bigl{<}X,t\bigr{>}\|_{L_{q}}\leq c_{2}\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}}=c_{2}\sqrt{q}.$$

$$\|\bigl{<}X,t\bigr{>}\|_{L_{q}}\leq L\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}}=L\sqrt{q}.$$

$$m\ge c_{5}s\log (ed/s),$$

$${\tilde{\Gamma }}_{ij}=\frac{x_{ij}}{{\left({\sum }_{\ell =1}^m{x}_{\ell j}^2\right)}^{1/2}}=\frac{{\Gamma }_{ij}}{\Vert \Gamma e_j{\Vert }_2}.$$

$$x=\epsilon \cdot \max \{1,\eta R\},$$

$$\|\bigl{<}X,t\bigr{>}\|_{L_{q}}\leq c_{0}L\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}}.$$

$$\Vert \sum _{j=1}^d{t}_j{x}_j{\Vert }_{L_p}\le L\Vert \sum _{t=1}^d{t}_j{z}_j{\Vert }_{L_p}.$$

$$\mathbb{E}(\sum _{j=1}^d{t}_j{x}_j{)}^p=\mathbb{E}\sum _{\vec{\beta }}{c}_{\vec{\beta }}\prod _{j=1}^d{t}_j^{{\beta }_j}{x}_j^{{\beta }_j}=\sum _{\vec{\beta }}{c}_{\vec{\beta }}\prod _{j=1}^d{t}_j^{{\beta }_j}\mathbb{E}{x}_j^{{\beta }_j},$$

$$\prod _{j=1}^d{t}_j^{{\beta }_j}\mathbb{E}{x}_j^{{\beta }_j}\le \prod _{j=1}^d{t}_j^{{\beta }_j}{L}^{{\beta }_j}\mathbb{E}{z}_j^{{\beta }_j}.$$

$$\sum _{\vec{\beta }}{c}_{\vec{\beta }}\prod _{j=1}^d{t}_j^{{\beta }_j}\mathbb{E}{x}_j^{{\beta }_j}\le L^p\sum _{\vec{\beta }}{c}_{\vec{\beta }}\prod _{j=1}^d{t}_j^{{\beta }_j}\mathbb{E}{z}_j^{{\beta }_j}=L^p\mathbb{E}(\sum _{j=1}^d{t}_j{z}_j{)}^p.$$

$$(\mathbb{E}|x{|}^q{)}^{1/q}\le 1+R{\delta }^{1/q}\le 2L\sqrt{q}\le c_{1}L(\mathbb{E}|g{|}^q{)}^{1/q}.$$

$$\Vert \sum _{j=1}^d{t}_j{x}_j{\Vert }_{L_q}\le c_{1}L\Vert |\sum _{j=1}^d{t}_j{g}_j{\Vert }_{L_q}\le c_{2}L\sqrt{q};$$

$$\Gamma =(x_{ij}{)}_{1\le i\le m,1\le j\le d}:{\mathbb{R}}^d\to {\mathbb{R}}^m.$$

$$\tilde{\Gamma}e_{j_{1}}=\frac{1}{(R^{2}+m-1)^{1/2}}\bigl{(}\varepsilon_{\ell j_{1}}Rf_{\ell}+\sum_{i\not=\ell}\varepsilon_{ij_{1}}f_{i}\bigr{)}$$

$$\tilde{\Gamma}e_{j_{2}}=\frac{1}{(R^{2}+m-1)^{1/2}}\bigl{(}\varepsilon_{\ell j_{2}}Rf_{\ell}+\sum_{i\not=\ell}\varepsilon_{ij_{2}}f_{i}\bigr{)}.$$

$$w_{\ell }=0\mathrm{and}{w}_i^2\le \frac{1}{R^2+m-1}\mathrm{for}i\ne \ell ;$$

$$\tilde{\Gamma }v\in \frac{\sqrt{m}}{R}{B}_2^m.$$

$${\Gamma }^J=(x_{ij}{)}_{1\le i\le m,j\in J}=({\epsilon }_{ij}{)}_{1\le i\le m,j\in J}$$

$${\tilde{\Gamma }}^J=({\tilde{\Gamma }}_{ij}{)}_{1\le i\le m,j\in J}=\frac{{\Gamma }^J}{\sqrt{m}}.$$

$$\frac{c}{\sqrt{m}}{B}_2^m\subset \frac{1}{\sqrt{m}}\Gamma B_1^J$$

$$\min \Vert t{\Vert }_1\mathrm{subject}\mathrm{to}\tilde{\Gamma }v=\tilde{\Gamma }t$$

$$\exists \ell \in \{1,...,m\}{\eta }_{\ell }=1\mathrm{and}{\eta }_i=0\mathrm{for}\mathrm{every}i\ne \ell .$$

$$m\delta (1-\delta {)}^{m-1}\ge \frac{2m}{d}$$

$${\eta }_i=0\mathrm{for}\mathrm{every}1\le i\le m$$

$$(1-\delta {)}^m\ge \frac{4m}{d},$$

$$c_4{B}_2^m\subset \Gamma B_1^J.$$

$$m\le c_2\sqrt{p}{d}^{1/p}.$$

$$\inf_{t\in\Sigma_{s}}\|\Gamma t\|_{2}^{2}=\inf_{t\in\Sigma_{s}}\sum_{i=1}^{m}\bigl{<}X_{i},t\bigr{>}^{2}\geq c_{3}(L)m\|t\|_{2}^{2}.$$

$$\underset{1\le j\le d}{\max }\Vert \Gamma e_j{\Vert }_2\le c_5(L)\sqrt{m}.$$

Full text

1 Introduction

Sparse Recovery is one of the most important research topics in modern signal processing. It focuses on the possibility of identifying a sparse signal—i.e., a signal that is supported on relatively few coordinates in $\mathbb{R}^{d}$ relative to the standard basis—using linear measurements. We refer the reader to the books [2, 3] for extensive surveys on sparse recovery and related topics.

In a basic sparse recovery problem one pre-selects an $m\times d$ matrix $\Gamma$ that generates the given data. For an unknown (sparse) vector $v$, the coordinates of the vector $\Gamma v$ are the $m$ linear measurements of $v$ one may use for recovery. The hope is that for a well chosen $\Gamma$, the resulting $m$ linear measurements would be enough to identify $v$, and because $v$ is sparse, the number of measurements required for recovery should be significantly smaller than the dimension $d$.

One of the main achievements of the theory of sparse recovery was the introduction of a convex optimization problem called basis pursuit, which is an effective recovery procedure: it selects $t\in\mathbb{R}^{d}$ that solves the minimization problem

[TABLE]

where we denote by $\|x\|_{p}=(\sum_{j=1}^{d}|x_{j}|^{p})^{1/p}$.

Extensive effort has been devoted to the question of finding conditions on the measurement matrix $\Gamma$ that ensure the recovery of any sparse vector; more accurately, one would like to guarantee that for every $s$-sparse vector $v$, the $\ell_{1}$ minimization problem (1.1) has a unique solution—$v$ itself.

Definition 1.1

Let $\Sigma_{s}$ be the set of $s$-sparse vectors in $\mathbb{R}^{d}$. A matrix $\Gamma:\mathbb{R}^{d}\to\mathbb{R}^{m}$ satisfies the exact reconstruction property of order $s$ if for every $v\in\Sigma_{s}$ there is a unique solution to the minimization problem (1.1) and that unique solution is $v$.

Because measurements are ‘expensive’, one would like to find matrices $\Gamma$ that satisfy the exact recovery property of order $s$ with the smallest number of measurements (rows) possible. One may show that if $\Gamma$ satisfies the exact reconstruction property of order $s$, then it must have at least $m\sim s\log(ed/s)$ rows. Moreover, typical realizations of various random matrices with $\sim s\log(ed/s)$ rows indeed satisfy the exact reconstruction property of order $s$ (see, e.g., [3]). Therefore, the optimal number of measurements required for the exact reconstruction property of order $s$ is $m\sim s\log(ed/s)$, and that number serves as the benchmark for an optimal measurement matrix.

The question we are interested in has to do with the normalization of the columns of the measurement matrix. It is often assumed in literature that the columns of $\Gamma$ have unit Euclidean norm (see, for example, [2] and [3] and references therein); i.e., if $\{e_{1},...,e_{d}\}$ is the standard basis in $\mathbb{R}^{d}$ then $\|\Gamma e_{j}\|_{2}=1$ for $1\leq j\leq d$. Column normalization appears frequently in various notions used in the study of the exact reconstruction property. Among these well-studied notions are coherence [3]; the restricted eigenvalues condition [1]; and the compatibility condition [2]. Moreover, in many real-world applications, measurement matrices with normalized columns tends to perform better than matrices whose columns have not been normalized.

While column normalization seems a natural idea, it adds substantial technical difficulties when studying random measurement matrices: normalizing the columns of a matrix with independent rows introduces additional dependencies. Despite the added difficulties, the results of [5] highlight the possibility that column normalization may still have a significant role to play in the context of random measurement matrices, particularly in heavy-tailed situations.

To formulate the results of [5] and explain their connection to column-normalization we need the following definition:

Definition 1.2

Let $x$ be a random variable. Given an integer $m\leq d$, let $(x_{ij})$, $1\leq i\leq m$, $1\leq j\leq d$ be $md$ independent copies of $x$. The random matrix generated by $x$ is $\Gamma=(x_{ij})_{1\leq i\leq m,1\leq j\leq d}$. Also, we denote by $X=(x_{j})_{j=1}^{d}$ a vector with $d$ independent copies of $x$; thus the rows of $\Gamma$ are $m$ independent copies of $X$.

The following result from [5] is a construction of random matrices generated by seemingly nice random variables, but despite that the matrices exhibit poor reconstruction properties.

Theorem 1.3

There exist absolute constants $c_{1},c_{2}$ and $c_{3}$ for which the following holds. For every $2<p\leq c_{1}\log d$ there is a mean-zero, variance one random variable $x$ that satisfies

$\bullet$* For every $2\leq q\leq p$ and every $t\in S^{d-1}$,*

[TABLE]

$\bullet$* If $m\leq c_{3}\sqrt{p}(d/\log d)^{1/p}$ then with probability $1/2$, $\Gamma$ does not satisfy the exact reconstruction property of order $1$.*

Theorem 1.3 implies that without assuming that each \bigl{<}X,t\bigr{>} has a subgaussian moment growth111Recall that a characterization of an $L$-subgaussian random variable is that $\|z\|_{L_{p}}\leq L\sqrt{p}\|z\|_{L_{2}}$ for every $p\geq 2$ up to the $p$-moment for $p$ close to $\log d$, the resulting measurement matrix is suboptimal. Indeed, under a modest assumption, say that \|\bigl{<}X,t\bigr{>}\|_{L_{4}}\leq c\|\bigl{<}X,t\bigr{>}\|_{L_{2}} for every $t\in\mathbb{R}^{d}$, the recovery of $1$-sparse vectors requires at least $(d/\log d)^{1/4}$ measurements. And, if $p=(\log d)/(\beta\log\log d)$ for $\beta$ large enough, then the number of measurements required for the recovery of $1$-sparse vectors is at least $\sim\log^{c\beta}d$, which is suboptimal when $c\beta>1$.

To put Theorem 1.3 in some perspective, it is complemented by a positive result, once linear forms have enough subgaussian moments.

Theorem 1.4

Let $x$ be a mean-zero, variance one random variable. Assume that for every $2\leq q\leq c_{4}\log d$ and every $t\in S^{d-1}$,

[TABLE]

If

[TABLE]

then with probability at least $1-1/d^{c_{6}}-2\exp(-c_{7}m)$, $\Gamma$ satisfies the exact reconstruction property of order $s$. Here, $c_{4}$ in an absolute constant and $c_{5},c_{6}$ and $c_{7}$ are constants that depend only on $L$.

It follows from Theorem 1.4 that if $X$ has a slightly better moment growth condition than in Theorem 1.3—a subgaussian growth up to $p\sim\log d$—the random measurement matrix generated by $x$ satisfies the exact reconstruction property of order $s$, for the optimal number of measurements $m\sim s\log(ed/s)$.

The connection with column-normalization arises from the main observation used in the proof of Theorem 1.4:

Lemma 1.5

Recall that $\Sigma_{s}$ denotes the set of $s$-sparse vectors in $\mathbb{R}^{d}$. Let $\Gamma:\mathbb{R}^{d}\to\mathbb{R}^{m}$. If

(a) $\|\Gamma x\|_{2}\geq\alpha\|x\|_{2}$ for every $x\in\Sigma_{s}$,

(b) $\|\Gamma e_{j}\|_{2}\leq\beta$ for every $j\in\{1,...,d\}$,

and $s_{1}=\lfloor\alpha^{2}(s-1)/(4\beta^{2})\rfloor-1$, then $\Gamma$ satisfies the exact reconstruction property of order $s_{1}$.

Lemma 1.5 gives a clear motivation for considering column-normalized random measurement matrices, and that motivation grows stronger when taking into account the proof of Theorem 1.4. It turns out that the ‘bottleneck’ in the proof is the upper bound on $\max_{1\leq j\leq d}\|\Gamma e_{j}\|_{2}$, while guaranteeing (a) requires a rather minimal small-ball assumption. Therefore, the seemingly more restrictive condition (a) is almost universally true (see [7, 5] for more details) and (b) is the only place in which the moment growth assumption is used in the proof of Theorem 1.4.

Clearly, column normalization resolves the issue of an upper estimate on $\max_{1\leq j\leq d}\|\Gamma e_{j}\|_{2}$. That, and the fact that (a) is true under minimal assumptions has led G. Lecué [4] to ask whether with column normalization, the moment growth condition (1.2) can be relaxed significantly, leading to a much stronger version of Theorem 1.4.

Question 1.6

Let $x$ be a mean-zero, variance $1$ random variable, set $\Gamma$ to be the $m\times d$ matrix generated by $x$ and let $\tilde{\Gamma}$ be the column-normalized matrix generated by $x$. Thus, the entries of $\tilde{\Gamma}$ are

[TABLE]

If \|\bigl{<}X,t\bigr{>}\|_{L_{4}}\leq L\|\bigl{<}X,t\bigr{>}\|_{L_{2}} for every $t\in\mathbb{R}^{d}$ and $m=c(L)s\log(ed/s)$, does $\tilde{\Gamma}$ satisfy the exact reconstruction property of order $s$ with high probability?

Our main result is a version of Theorem 1.3 for a column-normalized matrix generated by well chosen random variable, showing that the answer to question 1.6 is negative.

Theorem 1.7

There exist absolute constants $c_{1},c_{2}$ and $c_{3}$ for which the following holds. For every $2\leq p\leq\log d$ there is a symmetric, variance $1$ random variable $x$ with the following properties:

$\bullet$* If $x_{1},...,x_{d}$ are independent copies of $x$ and $X=(x_{j})_{j=1}^{d}$, then for every $t\in S^{d-1}$ and every $2\leq q\leq p$, \|\bigl{<}X,t\bigr{>}\|_{L_{q}}\leq c_{1}\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}}.*

$\bullet$* If $m\leq c_{2}\sqrt{p}d^{1/p}$, then with probability at least $1-2\exp(-c_{3}m)$, the $m\times d$ column-normalized matrix generated by $x$ does not satisfy the exact reconstruction property of order $2$.*

Theorem 1.7 answers Question 1.6 in the negative: column normalization does not improve the poor behaviour described in Theorem 1.3. Indeed, for $p=4$, linear forms \bigl{<}X,t\bigr{>} satisfy an $L_{2}-L_{4}$ norm equivalence, but the recovery of $2$-sparse vectors using $\tilde{\Gamma}$ requires at least $m\sim d^{1/4}$ measurements — significantly larger than the optimal number of measurements, $m\sim\log d$. Moreover, if $\beta>1$ and $p=(\log d)/{\beta\log\log d}$, then although \|\bigl{<}X,t\bigr{>}\|_{L_{q}}\lesssim\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}} for every $2\leq q\leq p$, the recovery of $2$-sparse vectors using $\tilde{\Gamma}$ requires at least $m\sim\log^{c\beta}d$ measurements, which, again, is suboptimal when $c\beta>1$.

Remark 1.8

Theorem 1.7 actually improves the estimates from Theorem 1.3: a logarithmic factor in the bound on the number of measurements is removed, and the probability estimate is significantly better: $1-2\exp(-cm)$ rather than constant probability.

Let us mention the straightforward observation that a version of Theorem 1.4 holds for column-normalized matrices as well.

Theorem 1.9

Let $x$ be and $L$ be as in Theorem 1.4 and let $\tilde{\Gamma}$ be the column-normalized measurement matrix generated by $x$. If $m\geq c_{8}(L)s\log(ed/s)$, then with probability at least $1-1/d^{c_{9}(L)}-2\exp(-c_{10}(L)m)$, $\tilde{\Gamma}$ satisfies the exact reconstruction property of order $s$.

Theorem 1.9 is an immediate consequence of the proof of Theorem 1.4; its proof is presented in Appendix A merely for the sake of completeness.

2 Proof of Theorem 1.7

Let $\varepsilon$ be a symmetric, $\{-1,1\}$-valued random variable, set $\eta$ to be a $\{0,1\}$-valued random variable with mean $\delta$ and let $R>0$; the values of $\delta$ and $R$ will be specified later. Let

[TABLE]

let $x_{1},...,x_{d}$ be independent copies of $x$ and set $X=(x_{1},...,x_{d})$.

Let us identify conditions under which $X$ satisfies the first part of Theorem 1.7.

Lemma 2.1

There exists an absolute constant $c_{0}$ for which the following holds. Assume that $\delta<1/2$ and that there is $L\geq 1$ such that for every $2\leq q\leq p$, $R\delta^{1/q}\leq L\sqrt{q}$. Then for every $t\in\mathbb{R}^{d}$ and every $2\leq q\leq p$,

[TABLE]

Moreover, for every $t\in\mathbb{R}^{d}$, \|\bigl{<}X,t\bigr{>}\|_{L_{2}}=c_{1}\|t\|_{2}, and $1/\sqrt{2}\leq c_{1}\leq 2L$.

In particular, $X/c_{1}$ is an isotropic random vector and for every $t\in\mathbb{R}^{d}$, \bigl{<}X,t\bigr{>} exhibits a $c_{0}L$-subgaussian moment growth up to the $p$-th moment.

The proof of Lemma 2.1 is based on a simple comparison argument:

Lemma 2.2

Let $x_{1},...,x_{d}$ be centred, independent random variables and assume $z_{1},...,z_{d}$ are also centred and independent. If $p$ is even and for every $1\leq j\leq d$ and every $1\leq q\leq p$, $\|x_{i}\|_{L_{q}}\leq L\|z_{i}\|_{L_{q}}$, then for every $t\in\mathbb{R}^{d}$,

[TABLE]

Proof.  By a standard symmetrization argument we may also assume that $z_{1},...,z_{d}$ and $x_{1},...,x_{d}$ are symmetric. Therefore,

[TABLE]

with the sum taken over all choices of $\vec{\beta}=(\beta_{1},...,\beta_{d})\in\{0,...,p\}^{d}$, where $\sum_{j=1}^{d}\beta_{j}=p$ and $c_{\vec{\beta}}$ is the appropriate multinomial coefficient. Since $x_{1},...,x_{d}$ are symmetric, the only products that do not vanish are when $\beta_{1},...,\beta_{d}$ are even, and if $\beta_{1},...,\beta_{d}$ are even then

[TABLE]

Therefore,

[TABLE]

Proof of Lemma 2.1. Observe that $x=\varepsilon\max\{1,R\eta\}$ is mean-zero and that $\mathbb{E}x^{2}=1\cdot(1-\delta)+R^{2}\delta$. Hence, if $\delta\leq 1/2$ and $R^{2}\delta\leq 2L^{2}$ then $1/2\leq\mathbb{E}x^{2}\leq 4L^{2}$—and the “moreover” part of the claim follows.

Turning to the first part of the claim, let $x_{1},...,x_{d}$ be independent copies of $x$, set $g$ to be a standard gaussian random variable and let $g_{1},...,g_{d}$ be independent copies of $g$. Recall that for every $2\leq q\leq p$, $R\delta^{1/q}\leq L\sqrt{q}$, and observe that

[TABLE]

Therefore, $(x_{1},...,x_{d})$ and $(g_{1},...,g_{d})$ satisfy the conditions of Lemma 2.2 with a constant $c_{1}L$. Applying Lemma 2.2, it follows that for every $t\in S^{d-1}$ and every $2\leq q\leq p$,

[TABLE]

thus, \|\bigl{<}X,t\bigr{>}\|_{L_{q}}\leq c_{3}L\sqrt{q}\|\bigl{<}X,t\bigr{>}\|_{L_{2}}.

The key part in the construction is the following lemma which describes the typical structure of the matrix generated by $x$,

[TABLE]

Lemma 2.3

There exist absolute constants $c_{1},c_{2},c_{3}$ and $c_{4}$ for which the following holds. Let $\delta=c_{1}/d$ and $R\geq c_{2}m$. Then, with probability at least $1-2\exp(-c_{3}m)$:

(1) there are indices $j_{1}\not=j_{2}\in\{1,...,d\}$ and $1\leq\ell\leq m$ such that $\eta_{\ell j_{1}}=\eta_{\ell j_{2}}=1$ and for $i\not=\ell$, $\eta_{\ell j_{1}}=\eta_{\ell j_{2}}=0$;

$(2)$* there is a subset $J\subset\{1,...,d\}$ of cardinality $|J|=2m$ such that $\eta_{ij}=0$ for every $j\in J$ and $1\leq i\leq m$;*

$(3)$* we have that $c_{4}B_{2}^{m}\subset\Gamma B_{1}^{J}$, where $B_{1}^{J}=\{x=\sum_{j\in J}x_{j}e_{j}:\|x\|_{1}\leq 1\}$ and $B_{2}^{m}=\{x\in\mathbb{R}^{m}:\|x\|_{2}\leq 1\}$.*

Corollary 2.4

If $\Gamma$ satisfies Lemma 2.3 then its column-normalized version $\tilde{\Gamma}$ does not satisfy the exact reconstruction property of order $2$.

Proof.  Using the notation of Lemma 2.3 and by its first part, $\|\Gamma e_{j_{1}}\|_{2}=\|\Gamma e_{j_{2}}\|_{2}=(R^{2}+m-1)^{1/2}$; hence, if we denote by $\{f_{1},...,f_{m}\}$ the standard basis of $\mathbb{R}^{m}$,

[TABLE]

and

[TABLE]

If $\varepsilon_{\ell j_{1}}\not=\varepsilon_{\ell j_{2}}$ set $v=(e_{j_{1}}+e_{j_{2}})/2$; otherwise, set $v=(e_{j_{1}}-e_{j_{2}})/2$. In either case, $v$ is $2$-sparse. Let $w=\tilde{\Gamma}v$ and observe that the coordinates of $w$ satisfy that

[TABLE]

therefore,

[TABLE]

Next, let $J$ be the set of coordinates given by the second part of Lemma 2.3. Clearly, $j_{1},j_{2}\not\in J$ and

[TABLE]

is an $m\times 2m$ Bernoulli matrix. Therefore,

[TABLE]

Observe that $\tilde{\Gamma}B_{1}^{J}=\tilde{\Gamma}^{J}B_{1}^{J}$ and by the third part of Lemma 2.3

[TABLE]

for an absolute constant $c$.

Hence, if $\sqrt{m}/R\leq c/\sqrt{m}$ then $\tilde{\Gamma}v\in\tilde{\Gamma}B_{1}^{J}$. Since $\|v\|_{1}=1$ and $v\not\in B_{1}^{J}$, it is evident that $v$ is not the unique solution of the minimization problem

[TABLE]

and $\tilde{\Gamma}$ does not satisfy the exact reconstruction property of order $2$.

The proof of Lemma 2.3 uses a standard fact on iid $\{0,1\}$-valued random variables: if $W_{1},...,W_{d}$ are independent copies of a $\{0,1\}$-valued random variable $W$ and $\mathbb{E}W=\mu$ then with probability at least $1-2\exp(-c\mu d)$, $\mu d/2\leq|\{j:W_{j}=1\}|\leq 3\mu d/2$.

Proof of Lemma 2.3. Let $\eta_{1},...,\eta_{m}$ be independent copies of $\eta$, let $Y$ be the indicator of the event

[TABLE]

Observe that $\mathbb{E}Y=m\delta(1-\delta)^{m-1}$ and that if $Y_{1},...,Y_{d}$ are independent copies of $Y$ and $\mathbb{E}Y\geq 2m/d$ then with probability at least $1-2\exp(-c_{1}m)$, $|\{i:Y_{i}=1\}|>m$. In particular, on that event, the matrix $(\eta_{ij})_{1\leq i\leq m,1\leq j\leq d}$ has at least two identical columns, each with a single entry of $1$. Therefore, the first part of Lemma 2.3 holds if

[TABLE]

For the second part of the lemma, let $Z$ be the indictor of the event

[TABLE]

and note that $\mathbb{E}Z=(1-\delta)^{m}$. If $Z_{1},...,Z_{d}$ are independent copies of $Z$ and $\mathbb{E}Z\geq 4m/d$ then with probability at least $1-2\exp(-c_{2}m)$, $|\{i:Z_{i}=1\}|\geq 2m$. Hence, if

[TABLE]

then with probability at least $1-2\exp(-c_{2}m)$, there is $J\subset\{1,...,d\}$ and for every $j\in J$ and every $1\leq i\leq m$, $\eta_{ij}=0$.

Turning to the third part of the lemma, and by applying the second part, we have that for $(i,j)\in\{1,...,m\}\times J$, $x_{ij}=\varepsilon_{ij}$. Let $\Gamma^{J}=(\varepsilon_{ij})_{1\leq i\leq m,j\in J}$ and recall that $(\varepsilon_{ij})$ are independent of $(\eta_{ij})$. Therefore, by Corollary 4.1 from [6], there are absolute constants $c_{3}$ and $c_{4}$ for which, with probability at least $1-2\exp(-c_{3}m)$,

[TABLE]

Finally, all that remains is to see when (2.1) and (2.2) are satisfied. It is straightforward to verify that if $\delta=2\theta/d$ for $2\leq\theta\leq c_{5}\frac{d}{m}\log(ed/m)$ then (2.1) holds, and if $\delta\leq\frac{c_{6}}{m}\log(ed/m)$ then (2.2) holds. Therefore, both conditions are satisfied with the choice of $\delta=c/d$ for a suitable absolute constant $c>1$, as long as $m\lesssim d$.

To complete the proof of Theorem 1.7, let $\delta=c_{1}/d$ as above, set $p>2$ and put $R=\sqrt{p}d^{1/p}$—a choice which complies with the conditions of Lemma 2.3 as long as

[TABLE]

It follows from Corollary 2.4 that with probability at least $1-2\exp(-c_{3}m)$, the column-normalized matrix $\tilde{\Gamma}$ generated by $x=\varepsilon\max\{1,R\eta\}$ does not satisfy the exact reconstruction property of order $2$. To complete that proof, all that remains is to show that $x$ also satisfies the assumptions of Lemma 2.1: that $R\delta^{1/q}\leq L\sqrt{q}$ for every $2\leq q\leq p$ and for an absolute constant $L$.

To that end, let $\phi(x)=\sqrt{x}(d/c_{1})^{1/x}$ and observe that $\phi(x)$ is decreasing when $2\leq x\leq 2\log(d/c_{1})$; hence, $\phi(p)/\phi(q)\leq 1$ for every $2\leq q\leq p$ as long as $p\leq 2\log(d/c_{1})$. Therefore, if we set $L=c_{1}$ then $R\delta^{1/q}\leq\sqrt{q}$ for every $q\leq p$, as required.

Appendix A Proof of Theorem 1.9

The proof is a direct consequence of the argument used in the proof of Theorem 1.4. Thanks to column normalization, $\tilde{\Gamma}$ satisfies (b) in Lemma 1.5 for $\beta=1$. All that is left to verify is (a) for $\alpha$ which is a constant that depends only on $L$.

The proof of Theorem 1.4 shows that if $\Gamma$ has $m\geq c_{1}(L)s\log(ed/s)$ independent rows that are distributed as $X$ then with probability at least $1-2\exp(-c_{2}(L)m)$,

[TABLE]

Also, with probability at least $1-1/d^{c_{4}(L)}$,

[TABLE]

For every $t\in\Sigma_{s}$, set

[TABLE]

which is also an $s$-sparse vector. Observe that $\tilde{\Gamma}t=\Gamma\tilde{t}$, implying that

[TABLE]

and (a) from Lemma 1.5 is verified for the matrix $\tilde{\Gamma}$ for $\alpha=c_{6}(L)$.