Invariance principle of random projection for the norm

TL;DR

This paper investigates how random projections preserve the distribution and concentration of the norm of high-dimensional random vectors with i.i.d. entries, extending Johnson-Lindenstrauss guarantees.

Contribution

It proves the asymptotic normality and concentration of the norm of random vectors under random projections, revealing low distortion effects.

Findings

Distribution of the norm is preserved under random projection.

Norm concentrates around its mean after projection.

Random matrices induce low distortion for high-dimensional random vectors.

Abstract

Johnson-Lindenstrauss guarantees certain topological structure is preserved under random projections when project high dimensional deterministic vectors to low dimensional vectors. In this work, we try to understand how random matrix affect norms of random vectors. In particular we prove the distribution of the norm of random vector $X \in \mathbb{R}^n$, whose entries are i.i.d. random variables, is preserved by random projection $S:\mathbb{R}^n \to \mathbb{R}^m$. More precisely, \[ \frac{X^TS^TSX - mn}{\sqrt{\sigma^2 m^2n+2mn^2}} \xrightarrow[\quad m/n\to 0 \quad ]{ m,n\to \infty } \mathcal{N}(0,1) \] We also prove a concentration of the random norm transformed by either random projection or random embedding. Overall, our results showed random matrix has low distortion for the norm of random vectors with i.i.d. entries.