Some Results on Regularization of LSQR and CGLS for Large-Scale Discrete Ill-Posed Problems

TL;DR

This paper analyzes the regularization properties of LSQR and CGLS for large-scale ill-posed problems, providing bounds on Krylov subspace accuracy and recommending hybrid methods for mildly ill-posed cases.

Contribution

It establishes quantitative bounds for Krylov subspace approximation of dominant singular vectors, advancing understanding of LSQR's regularization effects.

Findings

Krylov subspace captures dominant singular vectors in severely and moderately ill-posed problems.

Hybrid LSQR is recommended for mildly ill-posed problems.

Numerical experiments confirm theoretical bounds and phenomena.

Abstract

For large-scale discrete ill-posed problems, LSQR, a Lanczos bidiagonalization process based Krylov method, is most often used. It is well known that LSQR has natural regularizing properties, where the number of iterations plays the role of the regularization parameter. In this paper, for severely and moderately ill-posed problems, we establish quantitative bounds for the distance between the $k$-dimensional Krylov subspace and the subspace spanned by $k$ dominant right singular vectors. They show that the $k$-dimensional Krylov subspace may capture the $k$ dominant right singular vectors for severely and moderately ill-posed problems, but it seems not the case for mildly ill-posed problems. These results should be the first step towards to estimating the accuracy of the rank-$k$ approximation generated by Lanczos bidiagonalization. We also derive some other results, which help further understand the regularization effects of LSQR. We draw to a conclusion that a hybrid LSQR should generally be used for mildly ill-posed problems. We report numerical experiments to confirm our theory. We present more definitive and general observed phenomena, which will derive more research.