Matrix Factorizations with Uniformly Random Pivoting

TL;DR

This paper establishes a formal connection between matrix factorization algorithms and introduces a randomized pivoting rule that ensures linear convergence and stability, unifying analysis across different methods.

Contribution

It proposes a novel randomized pivoting rule applicable to a broad class of algorithms, providing a unified convergence analysis and stability guarantees.

Findings

Linear convergence rate for all algorithms under the new pivoting rule

Polynomial bound on numerical stability of Jacobi eigenvalue algorithm

Addresses longstanding open problem in numerical linear algebra

Abstract

This paper highlights a formal connection between two families of widely used matrix factorization algorithms in numerical linear algebra. One family consists of the Jacobi eigenvalue algorithm and its variants for computing the Hermitian eigendecomposition and singular value decomposition. The other consists of Gaussian elimination and the Gram-Schmidt procedure with various pivoting rules for computing the Cholesky decomposition and QR decomposition respectively. Both families are cast as special cases of a more general class of factorization algorithms. We provide a randomized pivoting rule that applies to this general class (which differs substantially from the usual pivoting rules for Gaussian elimination / Gram-Schmidt) which admits a unified analysis of the entire class of algorithms. The result is the same linear rate of convergence for each algorithm, irrespective of which factorization it computes. One important consequence of this randomized pivoting rule is a provable polynomial bound on the numerical stability of the Jacobi eigenvalue algorithm without any preconditioning, which addresses a longstanding open problem of Demmel and Veseli\'c `92.