Fast Dimensionality Reduction from $\ell_2$ to $\ell_p$

TL;DR

This paper introduces a fast linear embedding method for reducing dimensions from $\, ext{ell}_2$ to $\, ext{ell}_p$ for $p$ in [1,2], improving runtime for small target dimensions and establishing bounds for general norms.

Contribution

It generalizes $\, ext{ell}_2$ to $\, ext{ell}_p$ embeddings with improved runtime for small target dimensions and provides lower bounds for embeddings into arbitrary norms.

Findings

Achieves $O(d \,\log k)$ runtime for $k \,\leq d^{1/4}$

Extends $\, ext{ell}_2$ to $\, ext{ell}_p$ embeddings for $p \,\in [1,2]$

Establishes near-optimal lower bounds for embeddings into arbitrary norms.

Abstract

The Johnson-Lindenstrauss (JL) lemma is a fundamental result in dimensionality reduction, ensuring that any finite set $X \subseteq \mathbb{R}^d$ can be embedded into a lower-dimensional space $\mathbb{R}^k$ while approximately preserving all pairwise Euclidean distances. In recent years, embeddings that preserve Euclidean distances when measured via the $\ell_1$ norm in the target space have received increasing attention due to their relevance in applications such as nearest neighbor search in high dimensions. A recent breakthrough by Dirksen, Mendelson, and Stollenwerk established an optimal $\ell_2 \to \ell_1$ embedding with computational complexity $O(d \log d)$. In this work, we generalize this direction and propose a simple linear embedding from $\ell_2$ to $\ell_p$ for any $p \in [1,2]$ based on a construction of Ailon and Liberty. Our method achieves a reduced runtime of $O(d \log k)$ when $k \leq d^{1/4}$, improving upon prior runtime results when the target dimension is small. Additionally, we show that for \emph{any norm} $\|\cdot\|$ in the target space, any embedding of $(\mathbb{R}^d, \|\cdot\|_2)$ into $(\mathbb{R}^k, \|\cdot\|)$ with distortion $\varepsilon$ generally requires $k = \Omega\big(\varepsilon^{-2} \log(\varepsilon^2 n)/\log(1/\varepsilon)\big)$, matching the optimal bound for the $\ell_2$ case up to a logarithmic factor.