Solution of Least Squares Problems with Randomized Preconditioned Normal Equations

TL;DR

This paper demonstrates that using randomized preconditioners in solving least squares problems via normal equations yields solutions nearly as accurate as QR-based methods, even for ill-conditioned matrices, with probabilistic bounds supporting effectiveness.

Contribution

It introduces a randomized preconditioning approach for normal equations that maintains high solution accuracy and provides new perturbation bounds and probabilistic condition number estimates.

Findings

Preconditioned normal equations achieve near-QR accuracy.

Effective randomized preconditioners require minimal sampling.

Perturbation bounds match those of the original least squares problem.

Abstract

We consider the solution of full column-rank least squares problems by means of normal equations that are preconditioned, symmetrically or non-symmetrically, with a randomized preconditioner. With an effective preconditioner, the solutions from the preconditioned normal equations are almost as accurate as those from the QR-based Matlab backslash (mldivide) command -- even for highly illconditioned matrices. This means the accuracy of the preconditioned normal equations depends on the residual of the original least squares problem. We present non-intuitive but realistic perturbation bounds for the relative error in the computed solutions and show that, with an effective preconditioner, these bounds are essentially equal to the perturbation bound for the original least squares problem. Probabilitistic condition number bounds corroborate the effectiveness of the randomized preconditioner computed with small amounts of sampling.