Barriers for Faster Dimensionality Reduction

TL;DR

This paper establishes fundamental lower bounds on the embedding time for a broad class of Johnson-Lindenstrauss transforms, highlighting inherent computational limitations in fast dimensionality reduction algorithms.

Contribution

It provides the first non-trivial lower bounds of a9(m \u22c3 m) for a wide class of embedding algorithms, including many existing methods.

Findings

Lower bounds of a9(m d7 m) for embedding time

Most known fast JL transforms have embedding time a9(d d7 d)

Results suggest inherent computational limits for fast embeddings

Abstract

The Johnson-Lindenstrauss transform allows one to embed a dataset of $n$ points in $\mathbb{R}^d$ into $\mathbb{R}^m,$ while preserving the pairwise distance between any pair of points up to a factor $(1 \pm \varepsilon)$, provided that $m = \Omega(\varepsilon^{-2} \lg n)$. The transform has found an overwhelming number of algorithmic applications, allowing to speed up algorithms and reducing memory consumption at the price of a small loss in accuracy. A central line of research on such transforms, focus on developing fast embedding algorithms, with the classic example being the Fast JL transform by Ailon and Chazelle. All known such algorithms have an embedding time of $\Omega(d \lg d)$, but no lower bounds rule out a clean $O(d)$ embedding time. In this work, we establish the first non-trivial lower bounds (of magnitude $\Omega(m \lg m)$) for a large class of embedding algorithms, including in particular most known upper bounds.