Analysis of GMRES for Low-Rank and Small-Norm Perturbations of the Identity Matrix

TL;DR

This paper analyzes GMRES convergence for linear systems with matrices close to the identity plus low-rank and small-norm perturbations, providing bounds and insights into eigenvalue sensitivity and practical performance.

Contribution

It derives new bounds for GMRES residuals considering the pseudospectrum of the matrix and identifies eigenvalues sensitive to perturbations, extending understanding beyond classical cases.

Findings

GMRES convergence can be slow if sensitive eigenvalues are ill-conditioned.

At most 2p eigenvalues of the matrix are sensitive to small perturbations.

Numerical experiments demonstrate the theoretical bounds in PDE-related nonlinear systems.

Abstract

In many applications, linear systems arise where the coefficient matrix takes the special form ${\bf I} + {\bf K} + {\bf E}$, where ${\bf I}$ is the identity matrix of dimension $n$, ${\rm rank}({\bf K}) = p \ll n$, and $\|{\bf E}\| \leq \epsilon < 1$. GMRES convergence rates for linear systems with coefficient matrices of the forms ${\bf I} + {\bf K}$ and ${\bf I} + {\bf E}$ are guaranteed by well-known theory, but only relatively weak convergence bounds specific to matrices of the form ${\bf I} + {\bf K} + {\bf E}$ currently exist. In this paper, we explore the convergence properties of linear systems with such coefficient matrices by considering the pseudospectrum of ${\bf I} + {\bf K}$. We derive a bound for the GMRES residual in terms of $\epsilon$ when approximately solving the linear system $({\bf I} + {\bf K} + {\bf E}){\bf x} = {\bf b}$ and identify the eigenvalues of ${\bf I} + {\bf K}$ that are sensitive to perturbation. In particular, while a clustered spectrum away from the origin is often a good indicator of fast GMRES convergence, that convergence may be slow when some of those eigenvalues are ill-conditioned. We show there can be at most $2p$ eigenvalues of ${\bf I} + {\bf K}$ that are sensitive to small perturbations. We present numerical results when using GMRES to solve a sequence of linear systems of the form $({\bf I} + {\bf K}_j + {\bf E}_j){\bf x}_j = {\bf b}_j$ that arise from the application of Broyden's method to solve a nonlinear partial differential equation.