Arnoldi decomposition, GMRES, and preconditioning for linear discrete ill-posed problems

TL;DR

This paper analyzes why GMRES struggles with linear ill-posed problems and explores preconditioning and regularization techniques like Arnoldi-Tikhonov and Arnoldi-TSVD to improve solution stability and performance.

Contribution

It provides insights into GMRES limitations for ill-posed problems and introduces regularization-based variants using Arnoldi process for better solutions.

Findings

GMRES performance deteriorates on ill-posed problems without proper preconditioning.

Preconditioning and regularization improve convergence and stability.

Numerical examples demonstrate effectiveness of Arnoldi-based regularization methods.

Abstract

GMRES is one of the most popular iterative methods for the solution of large linear systems of equations that arise from the discretization of linear well-posed problems, such as Dirichlet boundary value problems for elliptic partial differential equations. The method is also applied to iteratively solve linear systems of equations that are obtained by discretizing linear ill-posed problems, such as many inverse problems. However, GMRES does not always perform well when applied to the latter kind of problems. This paper seeks to shed some light on reasons for the poor performance of GMRES in certain situations, and discusses some remedies based on specific kinds of preconditioning. The standard implementation of GMRES is based on the Arnoldi process, which also can be used to define a solution subspace for Tikhonov or TSVD regularization, giving rise to the Arnoldi-Tikhonov and Arnoldi-TSVD methods, respectively. The performance of the GMRES, the Arnoldi-Tikhonov, and the Arnoldi-TSVD methods is discussed. Numerical examples illustrate properties of these methods.