Analysis of Randomized Householder-Cholesky QR Factorization with Multisketching

TL;DR

This paper introduces a randomized QR algorithm, rand_cholQR, that achieves high stability comparable to shifted CholeskyQR3 while maintaining similar or better performance than CholeskyQR2 on GPUs, especially for tall-and-skinny matrices.

Contribution

The paper proposes and analyzes rand_cholQR, a randomized QR algorithm that improves stability without significant computational overhead, outperforming existing methods.

Findings

rand_cholQR is stable with high probability for full-rank matrices.

It performs nearly as fast or faster than CholeskyQR2 on GPU.

It offers improved stability over CholeskyQR2 with minimal additional cost.

Abstract

CholeskyQR2 and shifted CholeskyQR3 are two state-of-the-art algorithms for computing tall-and-skinny QR factorizations since they attain high performance on current computer architectures. However, to guarantee stability, for some applications, CholeskyQR2 faces a prohibitive restriction on the condition number of the underlying matrix to factorize. Shifted CholeskyQR3 is stable but has $50\%$ more computational and communication costs than CholeskyQR2. In this paper, a randomized QR algorithm called Randomized Householder-Cholesky (\texttt{rand\_cholQR}) is proposed and analyzed. Using one or two random sketch matrices, it is proved that with high probability, its orthogonality error is bounded by a constant of the order of unit roundoff for any numerically full-rank matrix, and hence it is as stable as shifted CholeskyQR3. An evaluation of the performance of \texttt{rand\_cholQR} on a NVIDIA A100 GPU demonstrates that for tall-and-skinny matrices, \texttt{rand\_cholQR} with multiple sketch matrices is nearly as fast as, or in some cases faster than, CholeskyQR2. Hence, compared to CholeskyQR2, \texttt{rand\_cholQR} is more stable with almost no extra computational or memory cost, and therefore a superior algorithm both in theory and practice.