A Hardware-aware and Stable Orthogonalization Framework

TL;DR

This paper introduces a hardware-aware orthogonalization framework using QR decomposition algorithms like CholeskyQR and TSQR, optimizing communication costs, stability, and adaptability across hardware architectures.

Contribution

It presents a novel framework that combines different orthogonalization algorithms tailored to hardware, improving efficiency and stability in Krylov space methods.

Findings

Reduced communication costs in orthogonalization

Maintained numerical stability with locally orthogonal representations

Validated performance improvements through numerical experiments

Abstract

The orthogonalization process is an essential building block in Krylov space methods, which takes up a large portion of the computational time. Commonly used methods, like the Gram-Schmidt method, consider the projection and normalization separately and store the orthogonal base explicitly. We consider the problem of orthogonalization and normalization as a QR decomposition problem on which we apply known algorithms, namely CholeskyQR and TSQR. This leads to methods that solve the orthogonlization problem with reduced communication costs, while maintaining stability and stores the orthogonal base in a locally orthogonal representation. Furthermore, we discuss the novel method as a framework which allows us to combine different orthogonalization algorithms and use the best algorithm for each part of the hardware. After the formulation of the methods, we show their advantageous performance properties based on a performance model that takes data transfers within compute nodes as well as message passing between compute nodes into account. The theoretic results are validated by numerical experiments.