Faster $p$-Norm Regression Using Sparsity

TL;DR

This paper introduces faster algorithms for p-norm regression problems that leverage sparsity and recent linear solver advances, achieving high accuracy solutions more efficiently than previous methods.

Contribution

It presents the first high-accuracy input sparsity p-norm regression algorithm for 1<p≤2, utilizing a new row sampling theorem and faster runtime complexities.

Findings

Achieves runtimes of ten(O(nnz(A) + d^4)) for 1<p

Leverages recent sparse linear solvers for faster algorithms

Provides the first high-accuracy input sparsity algorithms for p-norm regression

Abstract

For a matrix $A\in \mathbb{R}^{n\times d}$ with $n\geq d$, we consider the dual problems of $\min \|Ax-b\|_p^p, \, b\in \mathbb{R}^n$ and $\min_{A^\top x=b} \|x\|_p^p,\, b\in \mathbb{R}^d$. We improve the runtimes for solving these problems to high accuracy for every $p>1$ for sufficiently sparse matrices. We show that recent progress on fast sparse linear solvers can be leveraged to obtain faster than matrix-multiplication algorithms for any $p > 1$, i.e., in time $\tilde{O}(pn^\theta)$ for some $\theta < \omega$, the matrix multiplication constant. We give the first high-accuracy input sparsity $p$-norm regression algorithm for solving $\min \|Ax-b\|_p^p$ with $1 < p \leq 2$, via a new row sampling theorem for the smoothed $p$-norm function. This algorithm runs in time $\tilde{O}(\text{nnz}(A) + d^4)$ for any $1<p\leq 2$, and in time $\tilde{O}(\text{nnz}(A) + d^\theta)$ for $p$ close to $2$, improving on the previous best bound where the exponent of $d$ grows with $\max\{p, p/(p-1)\}$.