Randomized Iterative Solver as Iterative Refinement: A Simple Fix Towards Backward Stability

TL;DR

This paper introduces SIRR, a novel randomized least-squares solver that reorganizes iterative refinement to significantly improve backward stability and accuracy in large-scale problems.

Contribution

The paper presents SIRR, a new algorithm combining iterative and recursive refinement, achieving superior backward stability and accuracy in randomized least-squares solving.

Findings

SIRR achieves four orders of magnitude improvement in backward error.

SIRR is the first fast, single-stage randomized solver with both forward and backward stability.

Reorganizing computational order enhances numerical stability in sketching algorithms.

Abstract

Iterative sketching and sketch-and-precondition are well-established randomized algorithms for solving large-scale, over-determined linear least-squares problems. In this paper, we introduce a new perspective that interprets Iterative Sketching and Sketching-and-Precondition as forms of Iterative Refinement. We also examine the numerical stability of two distinct refinement strategies, iterative refinement and recursive refinement, which progressively improve the accuracy of a sketched linear solver. Building on this insight, we propose a novel algorithm, Sketched Iterative and Recursive Refinement (SIRR), which combines both refinement methods. SIRR demonstrates a \emph{four order of magnitude improvement} in backward error compared to iterative sketching, achieved simply by reorganizing the computational order, ensuring that the computed solution exactly solves a modified least-squares system where the coefficient matrix deviates only slightly from the original matrix. To the best of our knowledge, \emph{SIRR is the first asymptotically fast, single-stage randomized least-squares solver that achieves both forward and backward stability}.