Approximate Euclidean lengths and distances beyond Johnson-Lindenstrauss

TL;DR

This paper introduces a new algorithm that improves the estimation of Euclidean lengths and distances in high-dimensional data, reducing the dependency on the error parameter from quadratic to linear, especially for matrices with decaying spectra.

Contribution

It presents an algorithm inspired by Hutch++, that achieves better accuracy with fewer queries for length and distance estimation, extending to leverage scores, surpassing standard Johnson-Lindenstrauss bounds.

Findings

Achieves $oldsymbol{ ext{O}(rac{1}{ ext{epsilon}})}$ query complexity for length estimation.

Provides element-wise probabilistic bounds comparable or better than JL in worst-case.

Extends results to distance and leverage score estimation with similar improvements.

Abstract

A classical result of Johnson and Lindenstrauss states that a set of $n$ high dimensional data points can be projected down to $O(\log n/\epsilon^2)$ dimensions such that the square of their pairwise distances is preserved up to a small distortion $\epsilon\in(0,1)$. It has been proved that the JL lemma is optimal for the general case, therefore, improvements can only be explored for special cases. This work aims to improve the $\epsilon^{-2}$ dependency based on techniques inspired by the Hutch++ Algorithm, which reduces $\epsilon^{-2}$ to $\epsilon^{-1}$ for the related problem of implicit matrix trace estimation. We first present an algorithm to estimate the Euclidean lengths of the rows of a matrix. We prove for it element-wise probabilistic bounds that are at least as good as standard JL approximations in the worst-case, but are asymptotically better for matrices with decaying spectrum. Moreover, for any matrix, regardless of its spectrum, the algorithm achieves $\epsilon$-accuracy for the total, Frobenius norm-wise relative error using only $O(\epsilon^{-1})$ queries. This is a quadratic improvement over the norm-wise error of standard JL approximations. We also show how these results can be extended to estimate (i) the Euclidean distances between data points and (ii) the statistical leverage scores of tall-and-skinny data matrices, which are ubiquitous for many applications, with analogous theoretical improvements. Proof-of-concept numerical experiments are presented to validate the theoretical analysis.