GMRES on singular systems revisited

TL;DR

This paper revisits the convergence analysis of GMRES for singular systems, providing a complete proof of its behavior as a least squares solver under certain conditions, clarifying previous incomplete proofs.

Contribution

It offers a complete proof of GMRES's convergence properties for singular systems, specifically when the range of A equals the range of A transpose.

Findings

Provides a complete proof of GMRES convergence for singular systems

Clarifies the conditions under which GMRES yields least squares solutions

Addresses gaps in previous theoretical analyses

Abstract

In [Hayami K, Sugihara M. Numer Linear Algebra Appl. 2011; 18:449--469], the authors analyzed the convergence behaviour of the Generalized Minimal Residual (GMRES) method for the least squares problem $ \min_{ {\bf x} \in {\bf R}^n} {\| {\bf b} - A {\bf x} \|_2}^2$, where $ A \in {\bf R}^{n \times n}$ may be singular and $ {\bf b} \in {\bf R}^n$, by decomposing the algorithm into the range $ {\cal R}(A) $ and its orthogonal complement $ {\cal R}(A)^\perp $ components. However, we found that the proof of the fact that GMRES gives a least squares solution if $ {\cal R}(A) = {\cal R}(A^{\scriptsize T} ) $ was not complete. In this paper, we will give a complete proof.