On the geometry of polytopes generated by heavy-tailed random vectors

TL;DR

This paper investigates the geometry of random polytopes generated by heavy-tailed vectors, showing they contain a canonical body with high probability, and applies these findings to sparse recovery in compressive sensing.

Contribution

It introduces minimal assumptions on the generating vectors and establishes the presence of a canonical body in the polytopes, extending previous results to heavy-tailed distributions.

Findings

Random polytopes contain a deterministic polar of a floating body with high probability.

Established estimates for heavy-tailed random vectors like $q$-stable vectors.

Applied geometric results to noise blind sparse recovery in compressive sensing.

Abstract

We study the geometry of centrally-symmetric random polytopes, generated by $N$ independent copies of a random vector $X$ taking values in $\mathbb{R}^n$. We show that under minimal assumptions on $X$, for $N \gtrsim n$ and with high probability, the polytope contains a deterministic set that is naturally associated with the random vector---namely, the polar of a certain floating body. This solves the long-standing question on whether such a random polytope contains a canonical body. Moreover, by identifying the floating bodies associated with various random vectors we recover the estimates that have been obtained previously, and thanks to the minimal assumptions on $X$ we derive estimates in cases that had been out of reach, involving random polytopes generated by heavy-tailed random vectors (e.g., when $X$ is $q$-stable or when $X$ has an unconditional structure). Finally, the structural results are used for the study of a fundamental question in compressive sensing---noise blind sparse recovery.

Equations

$$c_1(\alpha )\sqrt{\log (eN/n)}{B}_2^n\subset \mathrm{absconv}(X_1,\dots ,X_N)$$

$$c_{2}(\alpha,L)\bigl{(}B_{\infty}^{n}\cap\sqrt{\log(eN/n)}B_{2}^{n}\bigr{)}\subset{\rm absconv}(X_{1},\ldots,X_{N}).$$

$$K_p(X):=\left\{t\in {\mathbb{R}}^n:\mathbb{P}(⟨X,t⟩\ge 1)\le \exp (-p)\right\}.$$

$$B(L_p(X)):=\{t\in {\mathbb{R}}^n:\Vert ⟨X,t⟩{\Vert }_{L_p}\le 1\}.$$

$$\Vert t{\Vert }_q={\left(\sum _{i=1}^n|t_i{|}^q\right)}^{1/q}and\; \Vert t{\Vert }_{\infty }=\underset{i=1,\dots ,n}{\max }|t_i|.$$

$$T^{\circ }=\{x\in {\mathbb{R}}^n:⟨t,x⟩\le 1\mathrm{for}\mathrm{every}t\in T\};$$

$$K_p(G)\sim (1/\sqrt{p})B_2^n,$$

$$K_p(\mathcal{E})\sim \mathrm{conv}(B_1^n\cup (1/\sqrt{p})B_2^n),$$

$${\rm absconv}(X_{1},\ldots,X_{N})\supset c_{1}\bigl{(}K_{p}(X)\bigr{)}^{\circ},$$

$$\mathcal{B}={\mathcal{B}}_{\Vert \cdot \Vert }=\{x\in {\mathbb{R}}^n:\Vert x\Vert \le 1\}\mathrm{and}\mathcal{S}={\mathcal{S}}_{\Vert \cdot \Vert }=\{x\in {\mathbb{R}}^n:\Vert x\Vert =1\}.$$

$$\mathbb{P}(|⟨X,t⟩|\ge \gamma \Vert t\Vert )\ge \delta .$$

$${\left(\mathbb{E}|⟨X,t⟩{|}^r\right)}^{1/r}\le L\Vert t\Vert .$$

$$\frac{1}{L}\mathcal{B}\subset B(L_r(X))\subset \frac{1}{\gamma {\delta }^{1/r}}\mathcal{B}.$$

$${\rm absconv}(X_{1},\ldots,X_{N})\supset\frac{1}{2}\bigl{(}K_{p}(X)\bigr{)}^{\circ},$$

$$\mathrm{absconv}(X_1,\dots ,X_N)\supset c_2(q){\left(\frac{N}{n}\right)}^{\alpha /q}{B}_{q^{\prime }}^n$$

$$\mathrm{absconv}(X_1,\dots ,X_N)\supset c_3{\left(\frac{N}{n}\right)}^{\alpha }{B}_{\infty }^n.$$

$$\underset{z\in {\mathbb{R}}^N}{\min }\Vert z{\Vert }_1\text{mbox}subjecttoAz=y,$$

$$\underset{z\in {\mathbb{R}}^N}{\min }\Vert z{\Vert }_1\text{mbox}subjectto\Vert Az-\bar{y}{\Vert }_2\le \eta$$

$$\Vert x-x^{♯}{\Vert }_1\lesssim {\sigma }_s(x{)}_1+\eta \sqrt{\frac{s}{n}},$$

$${\sigma }_s(x{)}_1=\underset{z:\Vert z{\Vert }_0\le s}{\inf }\Vert x-z{\Vert }_1$$

$$\Vert v{\Vert }_1\le D^{-1}\left|\left|\left|w\right|\right|\right|.$$

$$\Vert x^{♯}-x{\Vert }_1\lesssim {\sigma }_s(x{)}_1+\left|\left|\left|w\right|\right|\right|;$$

$${\left|\left|\left|x\right|\right|\right|}_p=\inf \{t>0:x\in t(K_p(X){)}^{\circ }\}.$$

$$\Vert v{\Vert }_1\le c_2^{-1}{\left|\left|\left|w\right|\right|\right|}_p,$$

$$K_p(X)=\left\{t\in {\mathbb{R}}^n,\mathbb{P}(⟨X,t⟩\ge 1)\le e^{-p}\right\}.$$

$$a{K}_p(X)=\left\{t\in {\mathbb{R}}^n,\mathbb{P}(⟨X,t⟩\ge a)\le e^{-p}\right\}.$$

$$\gamma \mathrm{absconv}(K_p(X))\subset \mathcal{B}.$$

$$\mathcal{B}\subset L\exp (p/r)K_p(X).$$

$$\mathbb{P}(⟨X,t⟩\ge \gamma )\ge \mathbb{P}(⟨X,t⟩\ge \gamma \Vert t\Vert )\ge \frac{\delta }{2}>\exp (-p),$$

$$\mathbb{P}(⟨X,u⟩\ge L\exp (p/r))\le \frac{\mathbb{E}|⟨X,u⟩{|}^r}{L^r}\exp (-p)\le \exp (-p),$$

Full text

1 Introduction

Let $X$ be a symmetric random vector in $\mathbb{R}^{n}$ and let $X_{1},\ldots,X_{N}$ be independent copies of $X$. The goal of this article is to study the geometry of the random polytope ${\rm absconv}(X_{1},\ldots,X_{N}),$ that is, the convex hull of the points $\pm X_{1},\ldots,\pm X_{N}$. Various aspects of the geometry of such random polytopes have been the subject of extensive study for many years. As a starting point, let us formulate two notable results in the direction we are interested in, and to that end, denote by $B_{p}^{n}$ the unit ball in $\ell_{p}^{n}$.

Theorem 1.1**.**

[18]** Let $X$ be the standard Gaussian random vector in $\mathbb{R}^{n}$, set $0<\alpha<1$ and consider $N\geq c_{0}(\alpha)n$. Then

[TABLE]

with probability at least $1-2\exp(-c_{2}N^{1-\alpha}n^{\alpha})$. Here $c_{0}$ and $c_{1}$ are constants that depend on $\alpha$ and $c_{2}$ is an absolute constant.

Theorem 1.1 can be extended beyond the Gaussian case, to random polytopes generated by a random vector $X=(\xi_{1},\ldots,\xi_{n})$ where the $\xi_{i}$’s are independent copies of a mean-zero, variance $1$ random variable $\xi$ that is $L$-subgaussian555Recall that a centered random variable is $L$-subgaussian if for every $p\geq 2$, $\left(\mathbb{E}|\xi|^{p}\right)^{1/p}\leq L\sqrt{p}$.. This class of random vectors includes, in particular, the Rademacher vector $X$, which is uniformly distributed in $\{-1,1\}^{n}$.

A version of Theorem 1.1 for the Rademacher vector was established in [17] (with a slightly suboptimal dependence of $N$ on the dimension $n$). The optimal estimate when $\xi$ is an arbitrary subgaussian random variable is the following special case of a result in [26].

Theorem 1.2**.**

[26]** Let $\xi$ be a mean-zero random variable that has variance $1$ and is $L$-subgaussian, and set $X=(\xi_{i})_{i=1}^{n}$ as above. Let $0<\alpha<1$ and set $N\geq c_{0}(\alpha,L)n$. Then there exists an absolute constant $c_{1}$ such that with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$

[TABLE]

In both cases, the typical random polytope ${\rm absconv}(X_{1},\ldots,X_{N})$ contains a large regular convex body: a multiple of the Euclidean unit ball when $X$ is the standard Gaussian random vector, and an intersection body of two $\ell_{p}$ balls when $X$ is $L$-subgaussian and has i.i.d. coordinates. As we explain in what follows, the fact that the bodies that are contained in ${\rm absconv}(X_{1},\ldots,X_{N})$ are different in these two examples is not a coincidence. Rather, it reflects the fact that a subgaussian random vector may in general generates a different geometry than the Gaussian one.

Motivated by these two facts, we study the following questions:

Question 1.3**.**

(1) Is it possible to find a set $K$ that is naturally associated with $X$ and is contained in ${\rm absconv}(X_{1},\ldots,X_{N})$ with high probability?

(2) If the answer to $(1)$ is yes, when does $K$ contain large (intersections of) $\ell_{p}$ balls as, for example, in Theorems 1.1 and 1.2?

Both Theorem 1.1 and Theorem 1.2, as well as the numerous other results in this direction, can be explained by a general principle stated in our main result, Theorem 1.6, that answers part (1) of Question 1.3. The geometric features of $X$ that are significant in this context are reflected by the natural floating bodies associated with $X$. Part (2) of Question 1.3 will be answered in Section 3 by identifying those floating bodies for a variety of choices of $X$—thus recovering, and at times improving, previously known results, as well as establishing new estimates in cases that were out of reach before.

Definition 1.4**.**

Let $X$ be a symmetric random vector in $\mathbb{R}^{n}$. For $p\geq 1$, we define the associated floating body

[TABLE]

The notion of floating bodies plays a crucial role in the study of approximation of convex bodies by polytopes, see, e.g., [36, 32, 3], where $X$ is distributed according to the uniform probability measure on the given convex body. It is known how to identify the floating bodies associated to Gaussian or Rademacher random vectors, see below.

In order to continue we require the following notation. Given sets $A,B\subset\mathbb{R}^{n}$, $A\sim B$ denotes that there are absolute constants $c_{1}$ and $c_{2}$ such that $c_{1}A\subset B\subset c_{2}B$. We write $A\sim_{\kappa}B$ if the constants $c_{1}$ and $c_{2}$ depend on the parameter $\kappa$. Identifying each $t\in\mathbb{R}^{n}$ with the linear functional $\left\langle\cdot,t\right\rangle$, we define, for $p>0$, the $L_{p}(X)$ (quasi-) norm of $t\in\mathbb{R}^{n}$ to be $\|\left\langle X,t\right\rangle\|_{L_{p}}=\left(\mathbb{E}|\left\langle X,t\right\rangle|^{p}\right)^{1/p}$, and denote its unit ball by

[TABLE]

For $1\leq q<\infty$ and $t=(t_{1},\ldots,t_{n})$, let

[TABLE]

For $1\leq q\leq\infty,$ let $B_{q}^{n}=\{t\in\mathbb{R}^{n}:\|t\|_{q}\leq 1\}$ be the unit ball of the normed space $\ell_{q}^{n}$, and set $q^{\prime}$ to be the conjugate index of $q$; that is, $\frac{1}{q}+\frac{1}{q^{\prime}}=1$. Finally, for $T\subset\mathbb{R}^{n}$ let

[TABLE]

the set $T^{\circ}$ is the polar body of $T$, which is a convex, centrally symmetric subset of $\mathbb{R}^{n}$ if $T$ is centrally symmetric.

With this notation in place, consider the following examples:

$\bullet$ Let $X=G$ be the standard Gaussian random vector in $\mathbb{R}^{n}$. Then for every $p\geq 1$,

[TABLE]

which can be shown by a direct calculation using the rotation invariance of $G$. Thus, the polar body of $K_{p}(G)$ satisfies $(K_{p}(G))^{\circ}\sim\sqrt{p}\,B_{2}^{n}$.

$\bullet$ Let $X={\cal E}$ be the Rademacher random vector in $\mathbb{R}^{n}$ (i.e., $X$ is uniformly distributed in $\{-1,1\}^{n}$). Results in [34] imply that

[TABLE]

and in particular, $(K_{p}({\cal E}))^{\circ}\sim B_{\infty}^{n}\cap\sqrt{p}B_{2}^{n}$.

Thus, the assertions of Theorem 1.1 for $X=G$ and of Theorem 1.2 for $X={\cal E}$ can be formulated in a unified way: with high probability, it holds that

[TABLE]

for $p=c_{2}\log(eN/n)$, where $c_{1}$ and $c_{2}$ are suitable constants. Our main result shows that this phenomenon holds under minimal assumptions on $X$, which we explain in the following.

Let $\|\cdot\|$ be a norm on $\mathbb{R}^{n}$ and set

[TABLE]

The random vector $X$ is said to satisfy a small-ball condition with respect to the norm $\|\cdot\|$ with constants $\gamma$ and $\delta$ if for every $t\in\mathbb{R}^{n}$,

[TABLE]

Also, for some $r>0$, $X$ is said to satisfy an $L_{r}$ condition with respect to the norm $\|\cdot\|$ and with constant $L$ if for every $t\in\mathbb{R}^{n}$,

[TABLE]

Assumption 1.5**.**

We assume that $X$ satisfies a small-ball condition with constants $\gamma>0$ and $\delta>0$, and an $L_{r}$ condition with constant $L$ for some $r>0$ with respect to the same norm $\|\cdot\|$.

Assumption 1.5 implies that the random vector $X$ is not degenerate: the small-ball condition (1.4) means that marginals of $X$ do not have ‘too much’ mass at [math], and the $L_{r}$ condition (1.5) leads to some minimal uniform control on the tail decay of each marginal. Also, it is straightforward to verify from (1.4) and (1.5) that

[TABLE]

Our answer to the first part of Question 1.3 is that under Assumption 1.5, a typical realization of ${\rm absconv}(X_{1},\ldots,X_{N})$ contains a constant multiple of $(K_{p}(X))^{\circ}$ for $p\sim\log(eN/n)$.

Theorem 1.6**.**

Let $X$ be a symmetric random vector that satisfies Assumption 1.5 with respect to a norm $\|\cdot\|$ and some $\delta,\gamma,r,L>0$. Let $0<\alpha<1$ and set $p=\alpha\log(eN/n)$ and assume that $N\geq c_{0}n$ for a constant $c_{0}=c_{0}(\alpha,\delta,r,L/\gamma)$. Let $X_{1},\ldots,X_{N}$ be independent copies of $X$ then with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$,

[TABLE]

where $c_{1}$ is an absolute constant.

Remark 1.7**.**

Theorem 1.6 still holds – even with the same constants and the same proof – when ${\rm absconv}(X_{1},\ldots,X_{N})$ is replaced by the standard convex hull $\mathop{\rm conv}(X_{1},\ldots,X_{N})$, see also Remark 2.6.

Assumption 1.5 is weaker than any of the assumptions in all previous results on the inner structure of ${\rm absconv}(X_{1},\ldots,X_{N})$. In particular, we allow heavy-tailed distributions and do not require independence of the entries of $X$. The freedom of choosing the norm $\|\cdot\|$ makes the method very flexible. Observe that (1.6) does not depend on the specific choice of $\|\cdot\|$, but the constant $c_{0}=c_{0}(\alpha,\delta,r,L/\gamma)$ does. In fact, the constants $L$ and $\gamma$ may change when chaining the norm. So the art consists in choosing a norm such that quotient $L/\gamma$, and hence, the constant $c_{0}$ become as small as possible.

As applications of Theorem 1.6 we show in Section 3 how one can recover or improve the previous central results on the geometry of the random polytope ${\rm absconv}(X_{1},\ldots,X_{N})$ in this context. This is done by answering the second part of Question 1.3: we identify the floating bodied $K_{p}(X)$ in all those cases, for example, when $X$ is the Gaussian vector ([18]); when $X$ has i.i.d. subgaussian centered coordinates ([26]); when $X$ is an isotropic, log-concave random vector ([10]); and when $X$ has i.i.d. centered coordinates that satisfy a small-ball condition ([19]).

In addition, and thanks to the universality of Theorem 1.6, one may establish various new outcomes that were previously completely out of reach like when $X$ is an unconditional random vector without necessarily independent entries, see Theorem 3.9. The main applications we present in this introduction are two results that we found to be particularly surprising: firstly, an answer to Question 1.3 when $X$ has i.i.d. $q$-stable coordinates for $1\leq q<2$ (e.g., a Cauchy random vector); and secondly, an answer to a fundamental question on sparse recovery.

1.1 Stable random vectors

Consider standard $q$-stable random vectors for $1\leq q<2$ (a $2$-stable random vector is just a Gaussian), that is, vectors that have i.i.d. standard $q$-stable random variables as coordinates. Recall that a random variable $\xi$ is standard $q$-stable if its characteristic function satisfies $\mathbb{E}[\exp(itX)]=\exp(-|t|^{q}/2)$ for every $t\in\mathbb{R}$ (we consider only the symmetric case). The following features of a standard $q$-stable random variable $\xi$ are of significance here:

$\bullet$ $\xi$ belongs to the weak-$L_{q}$ space; i.e., $\sup_{u>0}u^{q}\mathbb{P}(|\xi|>u)\leq C_{q}$, and for large values of $u$, $\mathbb{P}(|\xi|>u)\geq c_{q}/u^{q}$.

$\bullet$ the stability property: if $\xi_{1},\ldots,\xi_{n}$ are i.i.d. copies of $\xi$ and $t\in\mathbb{R}^{n}$ then $\sum_{i=1}^{n}t_{i}\xi_{i}$ has the same distribution as $\|t\|_{q}\,\xi$.

For a more comprehensive discussion on $q$-stable random variables see, e.g., [25, Chapter 5]. Note that for $q<2$, $\xi$ does not have a finite second moment, which makes the analysis of the structure of the random polytope ${\rm absconv}(X_{1},\ldots,X_{N})$ more challenging.

The answer to Question 1.3 for a $q$-stable random vector is as follows:

Theorem 1.8**.**

Let $\xi$ be a standard, $q$-stable random variable for some $1\leq q<2$. Let $\xi_{1},\ldots,\xi_{n}$ be independent copies of $\xi$ and set $X=(\xi_{i})_{i=1}^{n}$. Then for $0<\alpha<1$ and $N\geq c_{0}(\alpha,q)n$, with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$,

[TABLE]

*where $1/q+1/q^{\prime}=1$.

In particular, if $\xi$ is a standard Cauchy random variable (corresponding to $q=1$) then with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$*

[TABLE]

Observe that a typical realization of ${\rm absconv}(X_{1},\ldots,X_{N})$ is much larger than, say, the typical realization of the random polytope generated by the Gaussian random vector. Indeed, the latter only contains $\sqrt{\log(eN/n)}B_{2}^{n}$, which is a much smaller set than $c({N}/{n})^{\alpha/q}B_{q^{\prime}}^{n}$. The intuitive reason behind this phenomenon is that for $q<2$, a $q$-stable random variable is more ‘heavy-tailed’ than the Gaussian random variable: its tail decay is of the order of $u^{-q}$ rather than $\exp(-u^{2}/2)$ and that difference leads to the polynomial growth of the “inner radius” of ${\rm absconv}(X_{1},\ldots,X_{N})$. At the same time, the difference in the canonical body contained in ${\rm absconv}(X_{1},\ldots,X_{N})$ is due to the natural metric associated with $X$: each marginal $\left\langle t,X\right\rangle$ is distributed as $\|t\|_{q}\,\xi$ rather than as $\|t\|_{2}\,\xi$.

The proof of Theorem 1.8 is presented in Section 3.1.1.

1.2 Relation to Compressive Sensing

The second surprising outcome of Theorem 1.6 is related to a fundamental question in the area of compressive sensing666For more information on compressive sensing we refer the reader to [14, 9, 16], and for more a detailed explanation on the connections between the geometry of random polytopes and sparse recovery, see [12, 40, 9, 16, 15, 8].: can sparse signals be recovered efficiently when the given data consist of a few measurements that are noisy, but the ‘noise level’ is not known.

Suppose one would like to recover an unknown vector (signal) $x\in\mathbb{R}^{N}$ from an underdetermined set of a linear measurements, i.e., from $y=Ax\in\mathbb{R}^{n}$, where $A\in\mathbb{R}^{n\times N}$ with $n$ much smaller than $N$. While this is impossible in general, the theory of compressive sensing studies when such recovery is possible by efficient methods for ($s$-)sparse vectors, i.e., vectors in $\mathbb{R}^{N}$ that satisfy $\|x\|_{0}=|\{\ell:x_{\ell}\neq 0\}|\leq s\ll n$.

One of the main achievement of compressive sensing was the discovery that a computationally efficient recovery procedure can be used to recover the signal. Indeed, if $x^{\sharp}$ is the solution of the $\ell_{1}$-minimization problem

[TABLE]

then for a well-chosen $\sim s\log(eN/s)$ measurements, $x^{\sharp}$ coincides with the original $s$-sparse $x$. This upper estimate on the required number of measurements is optimal, and it is attained by a wide variety of random measurement ensembles—for example, if the measurements are $(\left\langle G_{i},x\right\rangle)_{i=1}^{n}$, i.e., $A$ is a draw of a random matrix with independent, mean-zero, variance one, Gaussian entries.

Naturally, to be of value in real-life applications, recovery should be possible in the presence of noise. The additional appeal of $\ell_{1}$-minimization is that it can be modified to perform well even if the given measurements $(\left\langle Ax,e_{i}\right\rangle)_{i=1}^{n}$ are corrupted by noise, and if the signal $x$ is not necessarily sparse but only approximately sparse in some appropriate sense. Indeed, assume that the data one is given is $\bar{y}=Ax+w$ for $x\in\mathbb{R}^{N}$ and a vector $w\in\mathbb{R}^{n}$ of perturbations (noise) with a known noise level $\|w\|_{2}\leq\eta$. It is important to emphasize that unlike standard problems in statistics, here $w$ is an arbitrary vector, rather than a random draw according to some statistical law.

One can show that for a variety of random matrices, a sample size of $n\sim s\log(eN/s)$ suffices to ensure that the minimizer $x^{\sharp}$ of the modified $\ell_{1}$-minimization problem

[TABLE]

satisfies

[TABLE]

where

[TABLE]

is the best $\ell_{1}$ approximation error of $x$ by an $s$-sparse vector; again, this is the best estimate one can hope for.

Unfortunately, the $\ell_{1}$-minimization procedure of (1.8) requires accurate information on the true noise level $\|w\|_{2}$, or at least a good upper estimate of it. However, in real world applications, this information is often not available. Getting the noise level wrong renders the estimate (1.9) useless: if the employed value of $\eta$ is an underestimation of the true noise level then the error bound (1.9) need not be valid. On the other hand, if $\eta$ is chosen to be significantly larger than the true noise level, the resulting error estimate (1.9) (involving the chosen $\eta$) is terribly loose.

As it happens, one can show that noise blind recovery, in which the noise level is not known, is possible if the measurement matrix $A$ satisfies two conditions:

(1) A version of the null space property (NSP), see (B.1). We refer the reader to [9, 16] for a detailed exposition on the NSP.

Identifying matrices that satisfy the null space property has been of considerable interest in recent years and many examples can be found, for example, in [2, 9, 16, 23, 31, 13]. From our perspective, and somewhat inaccurately put, it is important to note that the NSP is (almost) a necessary condition for sparse recovery in noise-free problems. Therefore, to have any hope of successful recovery in noisy problems, the measurement matrix has to satisfy some version of the NSP.

(2) The second, and seemingly more restrictive condition is the so-called $\ell_{1}$ quotient property  [12]. The matrix $A$ satisfies the $\ell_{1}$ quotient property with respect to the norm $\left|\!\left|\!\left|{\ }\right|\!\right|\!\right|$ if for every $w\in\mathbb{R}^{n}$ there exists a vector $v\in\mathbb{R}^{N}$ such that $Av=w$ and

[TABLE]

It follows that if $A$ satisfies an appropriate null space property and the $\ell_{1}$-quotient property, then the solution $x^{\sharp}$ of (1.7) for $y=Ax+w$ satisfies

[TABLE]

in other words, the noise-blind recovery error depends on ‘how far’ $x$ is from being sparse and on the norm $\left|\!\left|\!\left|{w}\right|\!\right|\!\right|$ of the noise vector. For the sake of completeness, an outline of the proof of (1.11) can be found in Appendix B.

Theorem 1.6 implies that contrary to prior belief, $(2)$ is not restrictive at all; in fact, it is almost universal. Indeed, let $\left|\!\left|\!\left|{\cdot}\right|\!\right|\!\right|_{p}$ be the norm whose unit ball is the polar body $(K_{p}(X))^{\circ}$, i.e.,

[TABLE]

Set $A=(X_{1}|\cdots|X_{N})$ to be the random matrix whose columns are independent random draws of the random vector $X$. Then the inclusion from Theorem 1.6 implies that for each vector $w\in\mathbb{R}^{n}$ there exists a vector $v\in\mathbb{R}^{N}$ such that $Av=w$ and

[TABLE]

which is precisely the $\ell_{1}$ quotient property with respect to the norm $\left|\!\left|\!\left|{\cdot}\right|\!\right|\!\right|$.

Thanks to the study of the floating bodies $K_{p}(X)$ presented in Section 3, the norm $\left|\!\left|\!\left|{\ }\right|\!\right|\!\right|_{p}$ can be identified in a variety of cases, and in some of which the appropriate null space property has already been established – leading the error bound (1.11). These examples include some of the natural random ensembles that are used in sparse recovery, for example, when $X$ has i.i.d. subgaussian or subexponential coordinates [2, 15]; when $X$ is an isotropic, log-concave random vector [2]; and when $X$ has independent coordinates that have $\log(N)$ finite moments [31, 13] (for example, when the coordinates are distributed according to the Student-$t$ distribution with $\sim\log N$ degrees of freedom).

Thanks to Theorem 1.6, the $\ell_{1}$ quotient property can be established in those (and many other) cases, implying that noise-blind recovery is possible. To give a flavour of such a result, we present the example of the Student-t distribution in an appendix. More information and numerical experiments are given in [22].

2 Proof of the main result

For the proof of Theorem 1.6, we need some basic properties of the floating body

[TABLE]

Recall that a set $K$ is star-shaped around [math] if for every $x\in K$ and any $0\leq\lambda\leq 1$, $\lambda x\in K$.

Proposition 2.1**.**

Let $X$ be a symmetric random vector on $\mathbb{R}^{n}$. Then

$(1)$* The set $K_{p}(X)$ is star-shaped and symmetric around [math]. Moreover, for any $a>0$,*

[TABLE]

$(2)$* Let $\|\cdot\|$ be a norm on $\mathbb{R}^{n}$ and denote its unit ball by ${\cal B}={\cal B}_{\|\cdot\|}=\{t\in\mathbb{R}^{n}:\|t\|\leq 1\}$. If $X$ satisfies the small-ball condition (1.4) with respect to the norm $\|\cdot\|$ with constants $\gamma$ and $\delta$, then for $p>\log(2/\delta)$,*

[TABLE]

$(3)$* If $X$ satisfies the $L_{r}$ condition (1.5) with respect to the norm $\|\cdot\|$ with constant $L$, then*

[TABLE]

Proof. The first observation is straightforward. To prove (2.1) observe that by convexity, it is enough to show that $\gamma K_{p}(X)\subset{\cal B}$. But if $\|t\|\geq 1$ then the small-ball condition and the symmetry of $X$ imply that

[TABLE]

provided that $\delta>2\exp(-p)$, as was assumed. Hence, $t\notin\gamma K_{p}(X)$.

As for (2.2), note that for $\|u\|\leq 1$, the $L_{r}$ condition yields that $\mathbb{E}|\left\langle X,u\right\rangle|^{r}\leq L^{r}$ and thus by Markov’s inequality

[TABLE]

hence, $u\in L\exp(p/r)K_{p}(X)$.

An outcome of Proposition 2.1 is that if $X$ satisfies Assumption 1.5 and

[TABLE]

then $K_{p}(X)$ is a centrally symmetric subset of $\mathbb{R}^{n}$ that is star-shaped around [math] and for which

[TABLE]

Let ${\cal S}$ be the unit sphere of $(\mathbb{R}^{n},\|\cdot\|)$. For $\theta\in{\cal S}$ set

[TABLE]

and note that by (2.4), $(1/L)\,\exp(-p/r)\leq r(\theta)\leq 1/\gamma$. With a possible abuse of notation, put

[TABLE]

Note that $\partial K_{p}(X)$ may not coincide with the topological boundary of $K_{p}(X)$ as $\theta\mapsto r(\theta)$ need not be continuous on ${\cal S}$ for general $X$.

Corollary 2.2**.**

For every $\theta\in{\cal S}$,

[TABLE]

Proof. It follows from the definition of $r(\theta)$ that for any $\rho>1$, $\rho r(\theta)\theta\not\in K_{p}(X)$, and thus,

[TABLE]

Taking the intersection of these events for any $\rho>1$ gives the result.

The proof of Theorem 1.6 follows the path set in the (much simpler) proof of Theorem 1.5 from [30]. The goal is to show that if $X$ satisfies Assumption 1.5, and

[TABLE]

then with probability at least

[TABLE]

one has that

[TABLE]

For a symmetric convex body $U$ with a nonempty interior, define its support function $h_{U}$ by

[TABLE]

The inclusion (2.4) ensures that $(K_{p}(X))^{\circ}$ has nonempty interior. Therefore, (2.6) is equivalent to

[TABLE]

for every $t\in\mathbb{R}^{n}$; and the negation of this event is that there exists $t\in\mathbb{R}^{n}$ such that

[TABLE]

By homogeneity of (2.7) and since $\theta\mapsto r(\theta)$ is bounded away from [math] on ${\cal S}$, it suffices to show that there is $t\in\partial K_{p}(X)$ for which (2.7) holds. Denote by $\Gamma:\mathbb{R}^{n}\to\mathbb{R}^{N}$ the random matrix whose rows are $X_{1},\ldots,X_{N}$. Observe that ${\rm absconv}(X_{1},\ldots,X_{N})=\Gamma^{*}B_{1}^{N}$, and therefore

[TABLE]

Moreover, for $t\in\partial K_{p}(X)$, the definition of polarity gives $\sup_{u\in(K_{p}(X))^{\circ}}\left\langle u,t\right\rangle\leq 1$. Hence, for the proof of Theorem 1.6 it remains to show that

[TABLE]

The proof of (2.9) is based on the small-ball method (see, for example, [29]). First, fix any $t\in\partial K_{p}(X)$ and recall that by Corollary 2.2,

[TABLE]

Therefore, by independence of the $X_{i}$ and Chernoff’s inequality, with probability at least

[TABLE]

it holds that

[TABLE]

Second, thanks to the high probability estimate (2.10), it follows from the union bound that if $T\subset\partial K_{p}(X)$ with

[TABLE]

then

[TABLE]

with probability at least

[TABLE]

The only restriction on the set $T\subset\partial K_{p}(X)$ is its cardinality. With this in mind, we will define $T$ as a covering of $\partial K_{p}(X)$ with balls of appropriate radius associated to the norm $\|\cdot\|$.

Observe that by (2.1), $\partial K_{p}(X)\subset(1/\gamma){\cal B}$ provided that $p>\log(2/\delta)$. By a standard volumetric estimate, see, e.g., [16, Proposition C.3], for every $\rho>0$ there exists a $\eta/\gamma$-cover of $\partial K_{p}(X)$ with respect to the norm $\|\cdot\|$ of cardinality at most $(1+2/\eta)^{n}$. This $\eta/\gamma$-cover has the required cardinality (2.12) if

[TABLE]

If

[TABLE]

then (2.14) is satisfied for the choice

[TABLE]

Denoting by ${\cal A}_{1}$ the event on which (2.13) holds for $T$ that is a minimal $\eta/\gamma$-cover of $\partial K_{p}(X)$, it is evident that

[TABLE]

Finally, for every $t\in\partial K_{p}(X)$ let $\pi t\in T$ be the nearest element to $t$ in the $(\eta/\gamma)$-cover with respect to the norm $\|\cdot\|$. Consider the event ${\cal A}_{2}$ on which

[TABLE]

For each $t\in\partial K_{p}(X)$ consider the sets of indices

[TABLE]

and observe that on the event ${\cal A}={\cal A}_{1}\cap{\cal A}_{2}$,

[TABLE]

Clearly,

[TABLE]

and therefore

[TABLE]

For each $t\in I(t):=I_{1}(t)\cap I_{2}^{c}(t)$ the triangle inequality gives

[TABLE]

In particular, on the event ${\cal A}$, it holds that $\inf_{t\in\partial K_{p}(X)}\|\Gamma t\|_{\infty}\geq\frac{1}{2}$ and

[TABLE]

Finally, let us show that $\mathbb{P}({\cal A}_{2})$ is ‘large enough’ for the right choice of $p$. To that end, observe that for every $t\in\partial K_{p}(X)$, $\|t-\pi t\|\leq(\eta/\gamma)$, and therefore,

[TABLE]

which is the supremum of an empirical process indexed by the class of indicator functions

[TABLE]

The wanted estimate on this supremum is based on an outcome of Talagrand’s concentration inequality for bounded empirical processes, in the special case in which the indexing class is binary-valued and has a finite Vapnik-Chervonenkis (VC) dimension (for a definition of the VC dimension, see, e.g., [39]).

Before stating this result, let us first recall the definition of VC dimension and a basic bound needed in our proof.

Definition 2.3**.**

Let ${\mathcal{F}}$ be a class of $\{0,1\}$-valued functions on a space $\Omega$. The class shatters $\{x_{1},\ldots,x_{k}\}\subset\Omega$, if for every $I\subset\{1,\ldots,k\}$ there exists a function $f_{I}\in{\mathcal{F}}$ for which $f_{I}(x_{i})=1$ if $i\in I$ and $f_{I}(x_{i})=0$ if $i\not\in I$. Let

[TABLE]

Lemma 2.4**.**

Let ${\cal D}$ be a set of subsets of $\Omega$ such that the set of indicator functions ${\mathcal{F}}=\{\mathbbm{1}_{D}:D\subset{\cal D}\}$ satisfies $VC({\mathcal{F}})=d$. If $\widetilde{{\mathcal{F}}}=\{\mathbbm{1}_{D\cup D^{\prime}}:D,D^{\prime}\in\mathcal{D}\}$ then $VC(\widetilde{{\mathcal{F}}})<10d$.

Proof.   The statement is a special case of [4, Lemma 3.2.3], which treats the case of the class of $k$ unions, i.e., ${\cal D}^{k}=\{D_{1}\cup\cdots\cup D_{k}:D_{1},\ldots,D_{k}\in{\cal D}\}$, and states that the VC dimension of the corresponding class ${\mathcal{F}}^{k}$ of indicator functions satisfies $VC({\mathcal{F}}^{k})<2dk\log_{2}(3k)$. For $k=2$ one has $VC(\widetilde{{\mathcal{F}}})=cd$ with $c=4\log_{2}(6)\approx 10.34$. The slightly better constant $10$ (or even $9.4$) follows from an inspection of the proof, which shows that a strict upper bound for the VC dimension of $\widetilde{{\mathcal{F}}}$ is any $m$ such that $(em/d)^{2d}<2^{m}$. An explicit calculation shows that $m=10d$ is a valid choice.

Let us now state the outcome of Talagrand’s concentration inequality when the indexing set of functions is a VC class (see [37] and also [28, Lemma 3.7]).

Theorem 2.5**.**

Let ${\mathcal{F}}$ be a class of $\{0,1\}$-valued functions for which $VC({\mathcal{F}})\leq d$ and $\sup_{f\in{\mathcal{F}}}\mathbb{E}f^{2}\leq\sigma^{2}$. Set

[TABLE]

where $c=8e^{2}\sqrt{2}\approx 83.6$. Then for any $x>0$,

[TABLE]

For the sake of completeness, we provide a sketch of the argument in Appendix A.

Let us return to the proof of Theorem 1.6 and consider ${\mathcal{F}}$ as defined in (2.18). Each $f\in{\mathcal{F}}$ is the indicator of a union of two half spaces in $\mathbb{R}^{n}$. By Radon’s theorem, the VC dimension of the class of indicators of half spaces in $\mathbb{R}^{n}$ is $n+1$, see e.g. [33, Theorem 3.4]. It follows from Lemma 2.4 that $d:=VC({\mathcal{F}})<10(n+1)$. Moreover, by the $L_{r}$ condition from Assumption 1.5 and Markov’s inequality, for any $\theta\in[0,1]$,

[TABLE]

Hence, any $\sigma^{2}\geq(2L\eta/\gamma)^{\theta r}$ is a valid choice in the context of Theorem 2.5. By our choice of $\eta$ in (2.16), this requirement is fulfilled for

[TABLE]

With that choice of $\sigma$ and $p=\alpha\log(eN/n)$ the first term in the definition (2.19) of $R$ can be bounded as

[TABLE]

Choosing $\theta=c_{1}\min\{1,1/r\}$ with $c_{1}=1/(16\cdot 80)$ and assuming $N\geq c_{2}n$ for a suitable constant $c_{2}=c_{2}(\alpha)$, it is evident that $\frac{640\log(c)(n+1)}{N}\leq\left(\frac{n}{eN}\right)^{\alpha}/16$ and therefore,

[TABLE]

Also, under the same assumptions, the second term in the definition (2.19) of $R$ can be estimated using (2.20) as

[TABLE]

provided that $N\geq c_{3}n$ for some suitable $c_{3}=c_{3}(\alpha,r,L/\gamma)$. Combining the two estimates, it follows that

[TABLE]

Moreover, with a similar argument we have that

[TABLE]

provided that $N\geq c_{4}n$ with $c_{4}=c_{4}(\alpha,r,L/\gamma)$; furthermore,

[TABLE]

Now, recall that we assumed (2.3), i.e., that $p>\log(2/\delta)$, which by definition of $p$ is equivalent to $N>e(2/\delta)^{1/\alpha}n$. At the same time, the requirement (2.15) is equivalent to $N\geq(16\ln(2)e^{\alpha})^{1/(1-\alpha)}n$.

Summarizing, all required conditions on $N$ are satisfied if $N\geq c_{0}n$ with

[TABLE]

In this case, choosing $x=\exp(-p)/16$ in Theorem 2.5 and noting that

[TABLE]

it is evident that

[TABLE]

outside an event whose probability is at most

[TABLE]

where $c_{6}=3/806$ and $c_{1}=c_{6}/e\approx 0.0014$. This completes the proof of (2.17) and, hence, of Theorem 1.6.

Remark 2.6**.**

The proof only needs very little adaptation if one replaces ${\rm{absconv}}(X_{1},\ldots,X_{N})$ by the standard convex hull $\mathop{\rm conv}(X_{1},\ldots,X_{N})$. In fact, $\mathop{\rm conv}(X_{1},\ldots,X_{N})=\Gamma^{*}\Delta_{1}^{N}$, where $\Delta_{1}^{N}=\{x\in\mathbb{R}^{N}:x_{i}\geq 0,\sum_{i=1}^{N}x_{i}\leq 1\}$ is the standard simplex. Then $\|\Gamma t\|_{\infty}$ in (2.8) and (2.9) is replaced by $\max_{i=1,\ldots,N}(\Gamma t)_{i}=\max_{i=1,\ldots,N}\langle X_{i},t\rangle$. Now, (2.11) works without the absolute values around $\langle X_{i},t\rangle$, anyway, so that the rest of the proof remains the same.

3 The floating bodies for various random vectors

Although Theorem 1.6 is (almost) universal, it is unrealistic to expect that the second part of Question 1.3 can be addressed with a single result. Therefore, the identity of the sets $K_{p}(X)$ has to be studied on a case-by-case basis. Having said that, there are some general principles that can be used to identify, or at least approximate the sets $K_{p}(X)$. Firstly, as outlined in what follows, there are natural examples in which $K_{p}(X)$ can be identified directly—among them are the standard Gaussian vector $X=G$; the standard Rademacher vector $X={\cal E}$; and when $X$ is a $q$-stable random vector. Secondly, we show in Section 3.2 that if linear forms $\left\langle X,t\right\rangle$ have $p$-th moments and satisfy a weak regularity condition, then $K_{p}(X)$ is equivalent to $B(L_{p}(X))$. Perhaps, one could have actually expected a variant of Theorem 1.6 with $K_{p}(X)$ replaced by $B(L_{p}(X))$ in the first place, but clearly $B(L_{p}(X))$ does not work in heavy-tailed situations, where it may be trivial if $p=\alpha\log(eN/n)>r$. This observation, combined with Theorem 1.6 improves the main result from [10] which studies random polytopes generated by isotropic, log-concave random vectors. Then, in Section 3.3, we explain how stochastic domination can be translated to information on the structures of the floating bodies. That allows one to show that ${\rm absconv}(X_{1},\ldots,X_{N})$ contains large canonical sets for very general random vectors, even when $X$ does not necessarily have independent entries.

3.1 Direct analysis of the floating body

The first two natural examples one should consider are $X=G$, the standard Gaussian random vector and $X={\cal E}$, the standard Rademacher random vector. A direct computation shows that

[TABLE]

and by [34],

[TABLE]

where $c_{1}$, $c_{1}^{\prime}$, $c_{2}$ and $c_{2}^{\prime}$ are absolute constants. Therefore, in both cases, Theorem 1.6 implies that ${\rm absconv}(X_{1},\ldots,X_{N})$ contains a large canonical body. In particular, one recovers the estimates of Theorem 1.1 and of Theorem 1.2 for the Rademacher random vector ${\cal E}$ stating that with high probability,

[TABLE]

and

[TABLE]

We explain how Theorem 1.2 can be recovered from Theorem 1.6 in full generality in Section 3.3.

Another, more surprising example in which $K_{p}(X)$ can be computed directly consists in the case that $X$ is a standard $q$-stable random vector, a situation outlined in Theorem 1.8.

3.1.1 Proof of Theorem 1.8

Recall that for $1\leq q<2$, a random variable $\xi$ is called standard $q$-stable if its characteristic function satisfies $\mathbb{E}[\exp(itX)]=\exp(-|t|^{q}/2)$ for every $t\in\mathbb{R}$ (we consider only the symmetric case). The proof of Theorem 1.8 is based on several well known facts, see, e.g., [25, Chapter 5].

(F$1)$ If $\xi_{1},\ldots,\xi_{n}$ are independent copies of a standard $q$-stable random variable $\xi$, and $X=(\xi_{i})_{i=1}^{n}$, then for any $t\in\mathbb{R}^{n}$, $\left\langle t,X\right\rangle$ has the same distribution as $\xi\|t\|_{q}$.

(F$2)$ While a standard $q$-stable random variable does not belong to $L_{q}$, it does belong to the weak-$L_{q}$ space $L_{q,\infty}$, i.e., $\sup_{u>0}u^{q}\mathbb{P}(|\xi|\geq u)\leq C_{q}$ for some constant $C_{q}>0$.

(F$3)$ The weak $L_{q}$ behaviour of $\xi$ is sharp: there exist constants $M_{q},c_{q}>0$ such that for any $u\geq M_{q}$, $\mathbb{P}(|\xi|\geq u)\geq c_{q}/u^{q}$.

From here on, let $\xi$ be a standard $q$-stable random variable for some $1\leq q<2$. Let us first show that $X$ satisfies Assumption 1.5, though obviously, due to the stability property (F$1)$, not with respect to the Euclidean norm, but rather with respect to $\|\cdot\|_{q}$. By (F$2)$, $\xi$ has a bounded $L_{r}$ (quasi)-norm for any $0<r<q$. As a result, $X$ satisfies the $L_{r}$ condition (1.5) with respect to $\|\cdot\|_{q}$ for $r=q/2$ and constant $L=L_{q}$. At the same time, e.g., by a Paley-Zygmund argument (see e.g. [11, Chapter 3.3]), it is straightforward to verify that $X$ satisfies the small-ball condition (1.4) with respect to $\|\cdot\|_{q}$ for constants $\gamma=\gamma_{q}$ and $\delta=\delta_{q}$ that depend only on $q$.

Therefore, invoking Theorem 1.6, a typical realization of ${\rm absconv}(X_{1},\ldots,X_{N})$ contains $c(K_{p}(X))^{\circ}$ for $p=\alpha\log(eN/n)$. It remains to identify the floating body $K_{p}(X)$. To this end, observe that

[TABLE]

Indeed, let $t\in K_{p}(X)$. By (F$1)$, $\left\langle X,t\right\rangle$ has the same distribution as $\xi\|t\|_{q}$ and

[TABLE]

Since $N/n$ is ‘large enough’, it follows that for $M_{q}$ as in (F$3)$, $\|t\|_{q}\leq 1/M_{q}$; indeed, otherwise $\mathbb{P}(\xi\geq M_{q})\leq(n/(eN))^{\alpha}$ which is impossible when $N/n$ is larger than a suitable constant. Now, by (F$3)$ ,

[TABLE]

implying that

[TABLE]

where $c_{2}=c_{2}(q)=c_{q}^{1/q}$. This establishes (3.1) and completes the proof of Theorem 1.8 by taking the polar.

3.2 Floating bodies and the unit ball of $L_{p}(X)$.

In order to get a better intuition on the role of the sets $K_{p}(X)$, let us consider a case in which $X$ is a ‘reasonably nice’ random vector, in the sense that each $\left\langle X,t\right\rangle$ has sufficiently many moments and exhibits a weak kind of regularity. As we show next, the sets $K_{p}(X)$ are then equivalent to

[TABLE]

The polar body

[TABLE]

is called the $L_{p}$-centroid body of $X$. The fact that there is a connection between $K_{p}(X)$ and $B(L_{p}(X))$ is an immediate outcome of Markov’s inequality:

[TABLE]

Therefore, if $\|\left\langle X,t\right\rangle\|_{L_{p}}\leq 1/e$ then $t\in K_{p}(X)$, i.e.,

[TABLE]

In order to prove a reverse inequality one requires an additional regularity condition on $X$.

Definition 3.1**.**

The random vector $X$ satisfies a regularity condition with constant ${D}$ if for every $q\geq 2$ and every $t\in\mathbb{R}^{n}$,

[TABLE]

Lemma 3.2**.**

Let $X$ be a symmetric random vector for which (3.4) holds. Then, for every $p\geq c_{2}$,

[TABLE]

where $c_{1}=1/(4\log(4{D}/3))$ and $c_{2}=\max\{2c_{1},2\log(2)\}$.

Proof.  Fix $t\in\mathbb{R}^{n}$. By the symmetry of $X$,

[TABLE]

and invoking the Paley-Zygmund inequality (see, e.g.[11, Chapter 3.3]) yields, for any $q\geq 2$,

[TABLE]

Hence, if $q=c_{1}p$ with $c_{1}=c_{1}({D})=1/(4\log(4{D}/3))$ and $p\geq 2\log(2)$ then

[TABLE]

Hence, if $t\in K_{p}(X)$ then $\|\left\langle X,t\right\rangle\|_{L_{q}}\leq 2$, as claimed.

Remark 3.3**.**

Note that in order to prove that $K_{p}(X)\subset 2B(L_{p}(X))$ for a fixed value of $p$ it suffices that $X$ satisfies that $\|\left\langle X,t\right\rangle\|_{L_{2q}}\leq{D}\|\left\langle X,t\right\rangle\|_{L_{q}}$ for $q=c^{\prime}p$.

Log-concave random vectors

Let us give one generic example in which (3.4) holds and $K_{p}(X)$ is equivalent to $B(L_{p}(X))$. There are many other natural examples of random vectors that satisfy (3.4) (e.g., the Rademacher vector ${\cal E}$, thanks to Borell’s hypercontractivity inequality [5]), but since the focus of this note is on random polytopes generated by a heavy-tailed random vectors we will not pursue this direction further.

A random vector is log-concave if it has a density $f$ satisfying that for every $x,y$ in its support and any $0<\lambda<1$, $f(\lambda x+(1-\lambda)y)\geq f^{\lambda}(x)f^{1-\lambda}(y)$. The $L_{p}$-centroid bodies $Z_{p}(X)$ defined in (3.2) play a crucial role in the study of log-concave measures [27, 35]. For more information on log-concave random vectors we refer the reader to [7, 20].

Let $X$ be a symmetric log-concave random vector that is non-degenerate, i.e., whose support is not contained in a proper subspace of $\mathbb{R}^{n}$. It follows from Borell’s inequality [5] (see e.g. [20, Proposition 5.16]) that for every $t\in\mathbb{R}^{n}$ and $1\leq p\leq q<\infty$,

[TABLE]

Therefore, $X$ satisfies the weak regularity condition (3.4) with constant ${D}=24$, implying that $B(L_{p}(X))\sim K_{p}(X)$. Further, by (3.5) with $q=2$, $X$ satisfies a small-ball condition with respect to the norm $\|t\|_{X}:=(\mathbb{E}|\langle X,t\rangle|^{2})^{1/2}=\|\Sigma^{1/2}t\|_{2}$ with constants $\gamma=1/2$ and $\delta=(1/32)^{4}$. Here $\Sigma=\mathbb{E}XX^{T}$ is the covariance matrix of $X$, which is nonsingular by the non-degenerateness assumption on $X$ so that $\|\cdot\|_{X}$ is actually a norm. Moreover, (3.6) also implies that $X$ satisfies the $L_{r}$-condition for $r=4$ with respect to $\|t\|_{X}$ with $L=24$. Theorem 1.6 then leads to the following result.

Theorem 3.4**.**

Let $X$ be a symmetric, non-degenerate, log-concave random vector. Let $0<\alpha<1$, set $N\geq c_{0}(\alpha)n$ and put $p=\alpha\log(eN/n)$. Then, with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$,

[TABLE]

where $c_{2}$ is a universal constant.

Theorem 3.4 improves the main result from [10], which states that if $X$ is an isotropic (which means that its covariance matrix $\Sigma$ is the identity), log-concave random vector and $\Gamma$ is the random matrix whose rows are $X_{1},\ldots,X_{N}$, then with probability at least $1-2\exp(-c_{1}(\alpha)N^{1-\alpha}n^{\alpha})-\mathbb{P}(\|\Gamma:\ell_{2}^{n}\to\ell_{2}^{N}\|\geq c\sqrt{N})$,

[TABLE]

Thanks to the progress made in [1] in the study of random matrices with i.i.d. isotropic log-concave rows, it is known that

[TABLE]

Therefore, the probability bound of the result in [10] is weaker than the one Theorem 3.4.

3.3 Stochastic domination

Up to this point, the examples focused on random vectors $X$ for which $K_{p}(X)$ can either be studied directly, or is equivalent to a natural convex body. One way of extending the scope of the analysis of the random polytopes ${\rm absconv}(X_{1},\ldots,X_{N})$ is by comparing the floating bodies $K_{p}(X)$ that are associated with different random vectors. As it happens, this comparison is simply a way of coding stochastic domination.

Definition 3.5**.**

Let $X$ and $Y$ be centered random vectors in $\mathbb{R}^{n}$. The random vector $X$ dominates $Y$ with constants $\lambda_{1}$ and $\lambda_{2}$ if for every $t\in\mathbb{R}^{n}$ and every $u>0$,

[TABLE]

This means that if $X$ dominates $Y$ with constants $\lambda_{1}$ and $\lambda_{2}$ then

[TABLE]

for $p^{\prime}=p-\log(1/\lambda_{1})$.

It is well known that this notion of domination is well-suited for the study of random vectors with i.i.d. coordinates because it is preserved under tensorization:

Theorem 3.6**.**

[24]** There are absolute constants $c_{1}$ and $c_{2}$ for which the following holds. Let $x$ and $y$ be symmetric random variables and assume that for every $u>0$, $\mathbb{P}(x>u)\geq\lambda_{1}\mathbb{P}(y\geq\lambda_{2}u)$. Let $x_{1},\ldots,x_{n}$ be independent copies of $X$ and set $y_{1},\ldots,y_{n}$ to be independent copies of $y$. Then $X=(x_{i})_{i=1}^{n}$ dominates $Y=(y_{i})_{i=1}^{n}$ with constants $c_{1}\lambda_{1}$ and $c_{2}\lambda_{2}$.

Theorem 3.6 leads to many structural results on ${\rm absconv}(X_{1},\ldots,X_{N})$ for vectors with i.i.d. coordinates, by comparing $x$ to a canonical random variable like a Rademacher random variable (i.e., a symmetric, $\{-1,1\}$-valued random variable) or to the standard Gaussian random variable.

Observe that if $x$ is a symmetric random variable that satisfies $\mathbb{P}(|x|\geq\gamma_{0})\geq\delta_{0}$ then we have

[TABLE]

where $\varepsilon$ is a Rademacher random variable. Hence, from Theorem 3.6, we get that if $x_{1},\ldots,x_{n}$ are independent copies of $x$ and $X=(x_{i})_{i=1}^{n}$, then $X$ dominates the Rademacher vector ${\cal E}$ with constants $\lambda_{1}$ and $\lambda_{2}$ that depend only on $\gamma_{0}$ and $\delta_{0}$. As a result, by (3.8),

[TABLE]

where $p^{\prime}=p-\log(1/\lambda_{1})$. Thanks to the characterization of $K_{p}({\cal E})$ and Theorem 1.6 one immediately recovers Theorem 1.2 as well as the main result from [19].

Theorem 3.7**.**

Let $x$ be a symmetric random variable that satisfies $\mathbb{E}x^{2}=1$ and set $x_{1},\ldots,x_{n}$ to be independent copies of $x$ and put $X=(x_{i})_{i=1}^{n}$. If there are constants $\gamma$ and $\delta$ such that $\mathbb{P}(|x|\geq\gamma)\geq\delta$, then for $N\geq c_{0}n$, with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$,

[TABLE]

here $c_{0}$ depends on $\alpha,\gamma$ and $\delta$, $c_{2}$ depends on $\gamma$ and $\delta$, and $c_{1}$ is an absolute constant.

The result can be pushed much further. The fact that $X$ has i.i.d. coordinates can be relaxed to an unconditional assumption. Moreover, $X$ need not have a covariance, as in fact, Assumption 1.5 suffices to get the desired conclusion.

Definition 3.8**.**

A random vector $X=(x_{i})_{i=1}^{n}$ is unconditional if for every $(\varepsilon_{i})_{i=1}^{n}\in\{-1,1\}^{n}$, $(x_{i})_{i=1}^{n}$ has the same distribution as $(\varepsilon_{i}x_{i})_{i=1}^{n}$.

Theorem 3.9**.**

For every $0<\delta<1$ there is a constant $c=c(\delta)$ such that the following holds. Let $X$ be an unconditional random vector that satisfies the small-ball condition with constants $\gamma$ and $\delta$. Then, for any $p>c_{0}(\delta)=4\log(8/\delta)+\log(4)$,

[TABLE]

In particular, if $X$ satisfies Assumption 1.5 and $N\geq c_{0}(\alpha,\delta,r,L/\gamma)n$, then with probability at least $1-2\exp(-c_{1}N^{1-\alpha}n^{\alpha})$,

[TABLE]

The proof of Theorem 3.9 is based on contraction inequalities for the Rademacher random vector (see, e.g. [25]): if $|a_{i}|\leq|b_{i}|$ for $1\leq i\leq n$ then for every $p\geq 1$,

[TABLE]

and for every $u>0$,

[TABLE]

We also require Borell’s hypercontractivity inequality [5]: for every $t\in\mathbb{R}^{n}$ and $q>p>1$,

[TABLE]

Proof of Theorem 3.9. The second part of the theorem is an immediate outcome of the first part, Theorem 1.6, and the fact that $X$ satisfies Assumption 1.5. To establish the first part, let us show that if $X=(x_{1},\ldots,x_{n})$ is an unconditional random vector and there are $\gamma,\delta>0$ such that for every $1\leq i\leq n$,

[TABLE]

then $K_{p}(X)\subset\frac{c(\delta)}{\gamma}K_{p}({\cal E}).$ Note that for this part of the theorem, $X$ does not need to satisfy the small-ball condition 1.4 for every direction, but rather only for coordinate directions.

Let $t\in K_{p}(X)$. Since $X$ is unconditional and symmetric, it holds that

[TABLE]

Let $\phi:\mathbb{R}\to\mathbb{R}_{+}$ be the truncation at level $\gamma$, that is,

[TABLE]

and set $Z_{t}=\sum_{i=1}^{n}\varepsilon_{i}\phi(x_{i})|t_{i}|$. Since $\phi(z)\leq|z|$ the contraction principle (3.10) yields, for every $(x_{i})_{i=1}^{n}\in\mathbb{R}^{n}$,

[TABLE]

Observe that for every $1\leq i\leq n$,

[TABLE]

where the last inequality follows from the small ball assumption (3.12). This observation implies that for any $q>1$,

[TABLE]

Here, the first inequality used that $\phi(z)\leq\gamma$ as well as the contraction principle (3.9), the second inequality is based on the hypercontractivity inequality for the Rademacher vector (3.11) and the last inequality follows from Jensen’s inequality. Therefore, by the Paley-Zygmund inequality (as in, e.g., [11, Chapter 3.3]), we have that

[TABLE]

If $q$ is such that

[TABLE]

then it follows that $\mathbb{E}|Z_{t}|^{q}<2$ because otherwise (3.16) would be in contraction to (3.14). Before elaborating on the implication of $\mathbb{E}|Z_{t}|^{q}<2$, let us discuss the particular choice

[TABLE]

Since $p>4\log(8/\delta)+\log(4)$ by assumption, it follows that $q>2$ and $q-1>q/2$ so that $(4q-2)/(q-1)<8-4/q<8$ and

[TABLE]

so that (3.17) is satisfied. Note that since $q>2$ and $p>4\log(8/\delta)+\log(4)$,

[TABLE]

By hypercontractivity combined with (3.15) (starting with the term after the second inequality in the first line) and the observation that $(\mathbb{E}|Z_{t}|^{q})^{1/q}<2^{1/q}<\sqrt{2}$, we obtain

[TABLE]

Markov’s inequality gives

[TABLE]

Hence, for

[TABLE]

it holds that $c(\delta)/\gamma=eC(\delta,\gamma)$ and $K_{p}(X)\subset\frac{c(\delta)}{\gamma}K_{p}({\cal E})$ as claimed.

Appendix A Concentration inequality for VC classes of functions

We prove Theorem 2.5 in this section, basically following [28] but with a simplification (avoiding the use of [28, Lemma 3.6] due to Talagrand [37]). The main tool is the following version of Talagrand’s concentration inequality [38] due to Bousquet [6], see also [16, Theorem 8.42], which features explicit and small constants.

Theorem A.1**.**

Let ${\mathcal{G}}$ be a set of functions $g:\mathbb{R}^{n}\to\mathbb{R}$. Let $X_{1},\ldots,X_{N}$ be independent random vectors in $\mathbb{R}^{n}$ such that $\mathbb{E}g(X_{\ell})=0$ and $|g(X_{\ell})|\leq K$ almost surely for all $\ell=1,\ldots,N$ and for all $g\in{\mathcal{F}}$ for some constant $K>0$. Introduce

[TABLE]

Let $\sigma^{2}_{\ell}>0$ such that $\mathbb{E}\left[g(X_{\ell})^{2}\right]\leq\sigma_{\ell}^{2}$ for all $g\in{\mathcal{G}}$ and $\ell=1,\ldots,N$. Then, for all $t>0$,

[TABLE]

where $\sigma^{2}_{\mathcal{G}}=\sum_{\ell=1}^{N}\sigma_{\ell}^{2}$.

In the situation of Theorem 2.5, we consider ${\mathcal{G}}=\{g=f-\mathbb{E}[f(X)]:f\in{\mathcal{F}}\}$, so that $\mathbb{E}g(X)=0$ and $|g(X)|\leq 1=:K$ almost surely for all $g\in{\mathcal{G}}$. Moreover, $\sigma_{\ell}^{2}\leq\sigma^{2}=\sup_{f\in{\mathcal{F}}}\mathbb{E}[f(X_{\ell})^{2}]$ so that $\sigma_{\mathcal{G}}^{2}\leq N\sigma^{2}$. It remains to estimate $\mathbb{E}Z$.

Symmetrization, see e.g. [25, Lemma 6.3], and Dudley’s inequality in the form of [16, Theorem 8.23] yield, for a Rademacher sequence $\varepsilon_{1},\ldots,\varepsilon_{N}$ independent of $X_{1},\ldots,X_{N}$,

[TABLE]

where the metric $d_{X,2}$ is given as

[TABLE]

and $\mathcal{N}({\mathcal{F}},d_{X,2},u)$ denote the covering numbers of ${\mathcal{F}}$, i.e., the minimal number of balls of radius $u$ in the metric $d_{X,2}$ required to cover ${\mathcal{F}}$ and

[TABLE]

where we have used that $f$ takes only values in $\{0,1\}$ in the equality. It follows from Haussler’s theorem [21] and the fact that the functions in ${\mathcal{F}}$ are $\{0,1\}$-valued (so that $\|f-g\|_{L_{2}(\mu)}^{2}=\|f-g\|_{L_{1}(\mu)}$ for any probability measure $\mu$) that the covering numbers can be estimated via the VC-dimension $d$ as

[TABLE]

Plugging this into our estimate of $\mathbb{E}Z$ above and noting that $(2e(d+1))^{1/2d}$ takes the maximum for $d=1$, so that $(2e(d+1))^{1/2d}\leq 2\sqrt{e}$ for all $d\geq 1$, gives

[TABLE]

We use the Cauchy-Schwarz inequality to estimate the integral

[TABLE]

Setting $\alpha=\sqrt{Y}/2$ and $\gamma=2e\sqrt{2}$, noting that $t\mapsto\sqrt{t\log(2e\sqrt{2}/t)}$ is concave and applying Jensen’s inequality gives

[TABLE]

Now observe that by the triangle inequality and since each $f$ takes values in $\{0,1\}$,

[TABLE]

Since $t\mapsto\sqrt{t\log(2e^{2}\sqrt{2}/t)}$ is increasing, this yields

[TABLE]

Setting $Q:=8\sqrt{d\log\left(\frac{8e^{2}\sqrt{2}}{\sigma^{2}}\right)}$ and squaring leads to the inequality $(\mathbb{E}Z)^{2}\leq Q^{2}(\mathbb{E}Z+N\sigma^{2})$ so that

[TABLE]

It follows from (A.2) that

[TABLE]

which is equivalent to the statement of Theorem (2.5).

Appendix B Sparse recovery

We begin this section with an outline of the proof of how the $\ell_{1}$-quotient property leads to (1.11). The null space property of $A$ of order $s$ with constant $\rho<1$ requiring that

[TABLE]

implies by [16, Theorem 4.12] that the solution $x^{\sharp}$ of equality constrained $\ell_{1}$-minimization (1.7) with $y=Ax$ satisfies

[TABLE]

If $y=Ax+w$, then the $\ell_{1}$-quotient property yields the existence of $v\in\mathbb{R}^{n}$ satisfying (1.10), so that we can write $y=A(x+v)$. The error bound (B.2) then leads to

[TABLE]

which is (1.11).

Next, let us turn to the example of noise-blind recovery when the measurement matrix has i.i.d. columns, selected according to the random vector $X$, which has i.i.d. coordinates, distributed according to the ($L_{2}$-normalized) Student-$t$ entries with $d=2\log N$ degrees of freedoms. In particular, the first moments of each coordinate are equivalent to that of a Gaussian random variable: for any $q\leq\log N$, $c_{1}\|g\|_{L_{q}}\leq\|\xi\|_{L_{q}}\leq c_{2}\|g\|_{L_{q}}$. This example is particularly interesting because it was recently shown (see, e.g., [31] and [13, Example 9]) that the corresponding random matrix satisfies the null space property (B.1) of order $s$ with high probability as long as $s\sim n/\log(eN/n)$. In addition, numerical tests in [13] show that this random matrix behaves precisely like a Gaussian random matrix in practical sparse recovery problems. However, the $\ell_{1}$-quotient property of a Student-$t$ matrix was previously open.

It is straightforward to verify that for any $q\leq\log N$ and every $w\in\mathbb{R}^{n}$, $\|\left\langle X,w\right\rangle\|_{L_{q}}\sim\|\left\langle G,w\right\rangle\|_{L_{q}}$. Moreover, setting $p=\alpha\log(eN/n)$, the results in Section 3.2 imply that

[TABLE]

therefore,

[TABLE]

The general error estimate (1.11) and Theorem 1.6 together with $s\sim n/\log(eN/n)$ lead to

[TABLE]

Note that (B.3) yields the same error estimate as (1.9) (up to absolute constants), but while (1.9) requires an a priori threshold for the noise level, (B.3) does not, and the error depends on the true noise level $\|w\|_{2}$ rather than a potentially pessimistic upper bound. We refer to [22] for more results in this direction and corresponding numerical experiments.

Acknowledgements

HR would like to thank the Isaac Newton Institute for Mathematical Science for support and hospitality during the program Approximation, Sampling and Compression in Data Science when work on this paper was undertaken. This work was supported by EPSRC Grant Number EP/R014604/1.

OG thanks the funding of the Fondation Simone et Cino Del Duca for the project ”Phénomènes en grande dimension”.

FK was supported by the German Science Foundations in the context of an Emmy Noether Junior Research Group (KR 4512/1-1)

AMS 2010 Classification:

primary: 52A22, 46B06, 60B20, 65K10 secondary: 52A23, 46B09, 15B52.

**Keywords: **

Random polytopes, random matrices, heavy tails, small ball probability, compressed sensing, $\ell_{1}$-quotient property.