On higher order isotropy conditions and lower bounds for sparse quadratic forms

TL;DR

This paper establishes lower bounds for empirical quadratic forms and compatibility constants in high-dimensional settings, leveraging higher order isotropy and moment conditions, with implications for compressed sensing and sparse estimation.

Contribution

It introduces new lower bounds for empirical quadratic forms under higher order isotropy, improving understanding of compatibility constants in high-dimensional statistics.

Findings

Empirical quadratic forms are bounded below by values approaching one.

Convergence of empirical compatibility constants to their theoretical values is demonstrated.

Lower bounds are obtained under minimal moment conditions when data is normalized.

Abstract

This study aims at contributing to lower bounds for empirical compatibility constants or empirical restricted eigenvalues. This is of importance in compressed sensing and theory for $\ell_1$-regularized estimators. Let $X$ be an $n \times p$ data matrix with rows being independent copies of a $p$-dimensional random variable. Let $\hat \Sigma := X^T X / n$ be the inner product matrix. We show that the quadratic forms $u^T \hat \Sigma u$ are lower bounded by a value converging to one, uniformly over the set of vectors $u$ with $u^T \Sigma_0 u $ equal to one and $\ell_1$-norm at most $M$. Here $\Sigma_0 := {\bf E} \hat \Sigma$ is the theoretical inner product matrix which we assume to exist. The constant $M$ is required to be of small order $\sqrt {n / \log p}$. We assume moreover $m$-th order isotropy for some $m >2$ and sub-exponential tails or moments up to order $\log p$ for the entries in $X$. As a consequence we obtain convergence of the empirical compatibility constant to its theoretical counterpart, and similarly for the empirical restricted eigenvalue. If the data matrix $X$ is first normalized so that its columns all have equal length we obtain lower bounds assuming only isotropy and no further moment conditions on its entries. The isotropy condition is shown to hold for certain martingale situations.