Fast randomized least-squares solvers can be just as accurate and stable as classical direct solvers

TL;DR

This paper introduces two backward stable randomized least-squares solvers, SPIR and FOSSILS, which match the accuracy and stability of classical methods while offering faster computation for overdetermined linear problems.

Contribution

The paper presents SPIR and FOSSILS, novel backward stable randomized algorithms that integrate iterative refinement with preconditioned methods for reliable least-squares solutions.

Findings

SPIR and FOSSILS achieve backward stability.

Both methods converge at rates comparable to existing randomized solvers.

They maintain accuracy and stability similar to classical QR-based solvers.

Abstract

One of the greatest success stories of randomized algorithms for linear algebra has been the development of fast, randomized algorithms for highly overdetermined linear least-squares problems. However, none of the existing algorithms is backward stable, preventing them from being deployed as drop-in replacements for existing QR-based solvers. This paper introduces sketch-and-precondition with iterative refinement (SPIR) and FOSSILS, two provably backward stable randomized least-squares solvers. SPIR and FOSSILS combine iterative refinement with a preconditioned iterative method applied to the normal equations and converge at the same rate as existing randomized least-squares solvers. This work offers the promise of incorporating randomized least-squares solvers into existing software libraries while maintaining the same level of accuracy and stability as classical solvers.