A simple tool for bounding the deviation of random matrices on geometric sets

TL;DR

This paper introduces a simple, general tool for bounding the deviation of random matrices on geometric sets, with applications in model selection and other areas, based on sub-gaussian properties and Gaussian complexity.

Contribution

The paper proves that the process involving the deviation of random matrices has sub-gaussian increments and provides bounds based on Gaussian complexity, including a local version for unbounded sets.

Findings

Deviation of $ orm{Ax}_2$ is uniformly bounded by Gaussian complexity.

The process $Z_x$ has sub-gaussian increments.

New results for model selection in constrained linear models.

Abstract

Let $A$ be an isotropic, sub-gaussian $m \times n$ matrix. We prove that the process $Z_x := \|Ax\|_2 - \sqrt m \|x\|_2$ has sub-gaussian increments. Using this, we show that for any bounded set $T \subseteq \mathbb{R}^n$, the deviation of $\|Ax\|_2$ around its mean is uniformly bounded by the Gaussian complexity of $T$. We also prove a local version of this theorem, which allows for unbounded sets. These theorems have various applications, some of which are reviewed in this paper. In particular, we give a new result regarding model selection in the constrained linear model.