Rigorous Restricted Isometry Property of Low-Dimensional Subspaces

TL;DR

This paper proves that Gaussian random projections preserve the distances between low-dimensional subspaces with high probability, providing a theoretical foundation for dimensionality reduction in subspace-based data analysis.

Contribution

It establishes a rigorous Restricted Isometry Property for low-dimensional subspaces under Gaussian random projections, extending RIP concepts beyond sparse vectors.

Findings

Projection Frobenius norm distances are preserved with high probability

Theoretical guarantees match the behavior of Johnson-Lindenstrauss lemma for subspaces

Supports dimensionality reduction in large-scale subspace data analysis

Abstract

Dimensionality reduction is in demand to reduce the complexity of solving large-scale problems with data lying in latent low-dimensional structures in machine learning and computer version. Motivated by such need, in this work we study the Restricted Isometry Property (RIP) of Gaussian random projections for low-dimensional subspaces in $\mathbb{R}^N$, and rigorously prove that the projection Frobenius norm distance between any two subspaces spanned by the projected data in $\mathbb{R}^n$ ($n<N$) remain almost the same as the distance between the original subspaces with probability no less than $1 - {\rm e}^{-\mathcal{O}(n)}$. Previously the well-known Johnson-Lindenstrauss (JL) Lemma and RIP for sparse vectors have been the foundation of sparse signal processing including Compressed Sensing. As an analogy to JL Lemma and RIP for sparse vectors, this work allows the use of random projections to reduce the ambient dimension with the theoretical guarantee that the distance between subspaces after compression is well preserved.