Faster Linear Algebra for Distance Matrices

TL;DR

This paper develops fast algorithms for linear algebra operations on distance matrices, achieving linear runtime for some metrics and establishing lower bounds for others, with broad applications in low-rank approximation and matrix multiplication.

Contribution

It introduces efficient algorithms for matrix-vector products on distance matrices, including the first linear-time algorithm for the  metric and lower bounds for  metrics, advancing the algorithmic understanding of distance matrices.

Findings

Linear-time algorithm for  distance matrix-vector multiplication.

 metric requires (n^2) time for matrix-vector multiplication.

Faster algorithms for low-rank approximation of distance matrices.

Abstract

The distance matrix of a dataset $X$ of $n$ points with respect to a distance function $f$ represents all pairwise distances between points in $X$ induced by $f$. Due to their wide applicability, distance matrices and related families of matrices have been the focus of many recent algorithmic works. We continue this line of research and take a broad view of algorithm design for distance matrices with the goal of designing fast algorithms, which are specifically tailored for distance matrices, for fundamental linear algebraic primitives. Our results include efficient algorithms for computing matrix-vector products for a wide class of distance matrices, such as the $\ell_1$ metric for which we get a linear runtime, as well as an $\Omega(n^2)$ lower bound for any algorithm which computes a matrix-vector product for the $\ell_{\infty}$ case, showing a separation between the $\ell_1$ and the $\ell_{\infty}$ metrics. Our upper bound results, in conjunction with recent works on the matrix-vector query model, have many further downstream applications, including the fastest algorithm for computing a relative error low-rank approximation for the distance matrix induced by $\ell_1$ and $\ell_2^2$ functions and the fastest algorithm for computing an additive error low-rank approximation for the $\ell_2$ metric, in addition to applications for fast matrix multiplication among others. We also give algorithms for constructing distance matrices and show that one can construct an approximate $\ell_2$ distance matrix in time faster than the bound implied by the Johnson-Lindenstrauss lemma.