Communication-optimal parallel and sequential QR and LU factorizations: theory and practice

TL;DR

This paper introduces communication-optimal algorithms for QR and LU factorizations that are efficient in both parallel and sequential settings, maintaining numerical stability comparable to traditional methods.

Contribution

It presents new algorithms, TSQR and CAQR, that minimize communication costs while preserving stability, applicable to various matrix layouts and sizes.

Findings

Algorithms are optimal up to polylogarithmic factors in communication.

TSQR is optimized for tall, skinny matrices in 1-D layouts.

CAQR extends TSQR to general rectangular matrices in 2-D layouts.

Abstract

We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform, and just as stable as Householder QR. Our first algorithm, Tall Skinny QR (TSQR), factors m-by-n matrices in a one-dimensional (1-D) block cyclic row layout, and is optimized for m >> n. Our second algorithm, CAQR (Communication-Avoiding QR), factors general rectangular matrices distributed in a two-dimensional block cyclic layout. It invokes TSQR for each block column factorization.