Implementation of QR factorization of tall and very skinny matrices on current GPUs

TL;DR

This paper explores efficient GPU implementations of QR decomposition for tall, skinny matrices, comparing methods like Gramian-based algorithms and TSQR, emphasizing optimization techniques to overcome memory bandwidth limitations.

Contribution

It introduces optimized GPU algorithms for QR factorization of tall-skinny matrices, including Q-less QR and shared memory exploitation, and compares their performance and complexity.

Findings

TSQR is competitive in time-to-solution.

Memory-bound regimes require specialized, optimized implementations.

Q-less QR reduces memory write-backs and improves performance.

Abstract

We consider the problem of computing a QR (or QZ) decomposition of a real, dense, tall and very skinny matrix. That is, the number of columns is tiny compared to the number of rows, rendering most computations completely or partially memory-bandwidth limited. The paper focuses on recent NVIDIA GPGPUs still supporting 64-bit floating-point arithmetic, but the findings carry over to AMD GPUs as well. We discuss two basic algorithms: Methods based on the normal equations (Gram matrix), in particular Cholesky-QR2 and SVQB, and the "tall-skinny QR" (TSQR), based on Householder transformations in a tree-reduction scheme. We propose two primary optimization techniques: Avoiding the write-back of the Q factor ("Q-less QR"), and exploiting fast local memory (shared memory on GPUs). We compare a straight-forward implementation of Gramian-based methods, and a more sophisticated TSQR implementation, in terms of performance achieved, time-to-solution, and implementation complexity. By performance modelling and numerical experiments with our own code and a vendor-optimized library routine, we demonstrate the crucial need for specialized methods and implementations in this memory-bound to transitional (memory/compute-bound) regime, and that TSQR is competitive in terms of time-to-solution, but at the cost of an investment in low-level code optimization.