On Sparsity and Sub-Gaussianity in the Johnson-Lindenstrauss Lemma

TL;DR

This paper offers a straightforward proof of the Johnson-Lindenstrauss lemma for sub-Gaussian variables and explores the impact of sparsity on the projection dimension, balancing computational efficiency and accuracy.

Contribution

It provides a simple proof emphasizing sub-Gaussianity and extends the analysis to sparse projections, highlighting the trade-offs involved.

Findings

Sub-Gaussian variables suffice for Johnson-Lindenstrauss projections.

Sparsity in projections reduces computational cost.

Sparsity limits the achievable target dimension.

Abstract

We provide a simple proof of the Johnson-Lindenstrauss lemma for sub-Gaussian variables. We extend the analysis to identify how sparse projections can be, and what the cost of sparsity is on the target dimension.The Johnson-Lindenstrauss lemma is the theoretical core of the dimensionality reduction methods based on random projections. While its original formulation involves matrices with Gaussian entries, the computational cost of random projections can be drastically reduced by the use of simpler variables, especially if they vanish with a high probability. In this paper, we propose a simple and elementary analysis of random projections under classical assumptions that emphasizes the key role of sub-Gaussianity. Furthermore, we show how to extend it to sparse projections, emphasizing the limits induced by the sparsity of the data itself.