Low-Rank Approximation by Randomly Pivoted LU

TL;DR

This paper introduces Randomly Pivoted LU (RPLU), a low-rank approximation method that converges rapidly for matrices with decaying singular values and outperforms existing algorithms in memory-limited and structured settings.

Contribution

The paper analyzes the convergence and efficiency of RPLU, a novel low-rank approximation algorithm based on randomized LU factorization with pivot sampling.

Findings

RPLU converges geometrically in expectation for certain matrices.

RPLU requires less memory and computational resources compared to existing methods.

RPLU performs well on structured matrices like Cauchy-like matrices and in applications such as rational approximation.

Abstract

The low-rank approximation properties of Randomly Pivoted LU (RPLU), a variant of Gaussian elimination where pivots are sampled proportional to the squared entries of the Schur complement, are analyzed. It is shown that the RPLU iterates converge geometrically in expectation for matrices with rapidly decaying singular values. RPLU outperforms existing low-rank approximation algorithms in two settings: first, when memory is limited, RPLU can be implemented with $\mathcal{O}(k^2 + m + n)$ storage and $\mathcal{O}( k(m + n)+ k\mathcal{M}(\mat{A}) + k^3)$ operations, where $\mathcal{M}(\mat{A})$ is the cost of a matvec with $\mat{A}\in\mathbb{C}^{n\times m}$ or its adjoint, for a rank-$k$ approximation. Second, when the matrix and its Schur complements share exploitable structure, such as for Cauchy-like matrices. The efficacy of RPLU is illustrated with several examples, including applications in rational approximation and solving large linear systems on GPUs.