Perturbation Analysis for Preconditioned Normal Equations in Mixed Precision

TL;DR

This paper analyzes the conditioning of various preconditioned normal equations in mixed precision, showing that randomized preconditioners can achieve high accuracy efficiently, especially suitable for GPU implementations.

Contribution

It provides a detailed perturbation analysis for preconditioned normal equations in mixed precision and proposes an automatic precision selection method based on condition number estimation.

Findings

Preconditioned normal equations have mild conditioning dependence on preconditioner quality.

Randomized preconditioners can match Matlab's mldivide accuracy.

The approach is efficient and suitable for GPU implementations.

Abstract

For real matrices of full column-rank, we analyze the conditioning of several types of normal equations that are preconditioned by a randomized preconditioner computed in lower precision. These include symmetrically preconditioned normal equations, half-preconditioned normal equations, seminormal equations and not-normal equations. Our perturbation bounds are realistic and informative, and suggest that the conditioning depends only mildly on the quality of the preconditioner; however, it does depend on the size of the least squares residual -- even if the normal equations do not originate from a least squares problem. We illustrate that a randomized preconditioner can deliver a solution accuracy comparable to that of Matlab's mldivide command, is efficient in practice, and well-suited to GPU implementations. For the computation of the preconditioner, we propose an automatic selection of the precision, based on a fast condition number estimation in lower precision.