Bulk Johnson-Lindenstrauss Lemmas

TL;DR

This paper extends the Johnson-Lindenstrauss lemma by showing that if only a fraction of distances are allowed to be distorted, a lower target dimension suffices, especially for data with certain stable rank properties.

Contribution

It introduces a bulk JL lemma that allows a fraction of distances to be distorted, reducing the target dimension based on stable rank and subset properties.

Findings

A target dimension of O(ε^{-2} log(4e/η) log(N)/R) suffices when only a fraction η of distances are distorted.

Stable rank of matrices influences the dimension reduction bounds.

Refined results are provided for i.i.d. random vectors, especially when isotropic.

Abstract

For a set $X$ of $N$ points in $\mathbb{R}^D$, the Johnson-Lindenstrauss lemma provides random linear maps that approximately preserve all pairwise distances in $X$ -- up to multiplicative error $(1\pm \epsilon)$ with high probability -- using a target dimension of $O(\epsilon^{-2}\log(N))$. Certain known point sets actually require a target dimension this large -- any smaller dimension forces at least one distance to be stretched or compressed too much. What happens to the remaining distances? If we only allow a fraction $\eta$ of the distances to be distorted beyond tolerance $(1\pm \epsilon)$, we show a target dimension of $O(\epsilon^{-2}\log(4e/\eta)\log(N)/R)$ is sufficient for the remaining distances. With the stable rank of a matrix $A$ as $\lVert{A\rVert}_F^2/\lVert{A\rVert}^2$, the parameter $R$ is the minimal stable rank over certain $\log(N)$ sized subsets of $X-X$ or their unit normalized versions, involving each point of $X$ exactly once. The linear maps may be taken as random matrices with i.i.d. zero-mean unit-variance sub-gaussian entries. When the data is sampled i.i.d. as a given random vector $\xi$, refined statements are provided; the most improvement happens when $\xi$ or the unit normalized $\widehat{\xi-\xi'}$ is isotropic, with $\xi'$ an independent copy of $\xi$, and includes the case of i.i.d. coordinates.