Superlinear Convergence of GMRES for clustered eigenvalues and its application to least squares problems

TL;DR

This paper analyzes the superlinear convergence of GMRES for matrices with clustered eigenvalues, explaining the phenomenon through eigenvalue and eigenvector properties, and applies findings to least squares problems.

Contribution

It provides a theoretical explanation for GMRES superlinear convergence with clustered eigenvalues and extends the analysis to preconditioned GMRES in least squares problems.

Findings

Superlinear convergence occurs when eigenvalues are clustered and the right-hand side vector is well aligned with eigenvectors.

Eigenvalues alone do not fully explain convergence; eigenvectors and the right-hand side vector are crucial.

The analysis applies to preconditioned GMRES for least squares, explaining convergence behavior.

Abstract

The objective of this paper is to understand the superlinear convergence behavior of the GMRES method when the coefficient matrix has clustered eigenvalues. In order to understand the phenomenon, we analyze the convergence using the Vandermonde matrix which is defined using the eigenvalues of the coefficient matrix. Although eigenvalues alone cannot explain the convergence, they may provide an upper bound of the residual, together with the right hand side vector and the eigenvectors of the coefficient matrix. We show that when the coefficient matrix is diagonalizable, if the eigenvalues of the coefficient matrix are clustered, the upper bound of the convergence curve shows superlinear convergence, when the norm of the matrix obtained by decomposing the right hand side vector into the eigenvector components is not so large. We apply the analysis to explain the convergence of inner-iteration preconditioned GMRES for least squares problems.