Single-pass Nystr\"{o}m approximation in mixed precision

TL;DR

This paper analyzes the stability and accuracy of a single-pass Nyström method for positive semidefinite matrices in mixed precision, providing guidelines for precision choice and applications in preconditioning.

Contribution

It offers a rigorous error analysis of the single-pass Nyström method in mixed precision and develops heuristics for precision selection based on approximation rank.

Findings

The analysis confirms lower precision can be used for smaller rank approximations.

The method effectively constructs preconditioners for conjugate gradient.

Numerical experiments validate the stability and efficiency of the approach.

Abstract

Low rank matrix approximations appear in a number of scientific computing applications. We consider the Nystr\"{o}m method for approximating a positive semidefinite matrix $A$. In the case that $A$ is very large or its entries can only be accessed once, a single-pass version may be necessary. In this work, we perform a complete rounding error analysis of the single-pass Nystr\"{o}m method in two precisions, where the computation of the expensive matrix product with $A$ is assumed to be performed in the lower of the two precisions. Our analysis gives insight into how the sketching matrix and shift should be chosen to ensure stability, implementation aspects which have been commented on in the literature but not yet rigorously justified. We further develop a heuristic to determine how to pick the lower precision, which confirms the general intuition that the lower the desired rank of the approximation, the lower the precision we can use without detriment. We also demonstrate that our mixed precision Nystr\"{o}m method can be used to inexpensively construct limited memory preconditioners for the conjugate gradient method and derive a bound the condition number of the resulting preconditioned coefficient matrix. We present numerical experiments on a set of matrices with various spectral decays and demonstrate the utility of our mixed precision approach.