On the backward stability of s-step GMRES

TL;DR

This paper investigates the backward stability of s-step GMRES, a communication-avoiding Krylov subspace method, providing a new analysis framework, identifying instability issues, and proposing a modified Arnoldi process for improved stability and larger block sizes.

Contribution

The paper introduces an improved analysis framework for s-step GMRES that isolates rounding errors, and proposes a modified Arnoldi process to enhance stability and allow larger block sizes.

Findings

Analysis framework clarifies the impact of orthogonalization methods on backward error.

Classical s-step Arnoldi process can cause instability in GMRES.

Modified s-step Arnoldi process improves stability and allows larger block sizes.

Abstract

Communication, i.e., data movement, is a critical bottleneck for the performance of classical Krylov subspace method solvers on modern computer architectures. Variants of these methods which avoid communication have been introduced, which, while equivalent in exact arithmetic, can be unstable in finite precision. In this work, we address the backward stability of $s$-step GMRES, also known as communication-avoiding GMRES. Compared to the ``modular framework'' proposed in [A.~Buttari, N.~J.~Higham, T.~Mary, \& B.~Vieubl\'e. Preprint in 2024.], we present an improved framework for simplifying the analysis of $s$-step GMRES, which includes standard GMRES ($s=1$) as a special case, by isolating the effects of rounding errors in the QR factorization and the solution of the least squares problem. The key advantage of this new framework is that it is evident how the orthogonalization method affects the backward error, and it is not necessary to re-evaluate anything other than the orthogonalization itself when modifying the orthogonalization used in GMRES. Using this framework, we analyze $s$-step GMRES with popular block orthogonalization methods: block modified Gram--Schmidt and reorthogonalized block classical Gram--Schmidt algorithms. An example illustrates the resulting instability of $s$-step GMRES when paired with the classical $s$-step Arnoldi process and shows the limitations of popular strategies for resolving this instability. To address this issue, we propose a modified $s$-step Arnoldi process that allows for much larger block size $s$ while maintaining satisfactory accuracy, as confirmed by our numerical experiments.