Hybrid CG-Tikhonov is a filtration of the CG Lanczos vectors

TL;DR

This paper introduces a new interpretation of hybrid CG-Tikhonov regularization as a filtration of Lanczos vectors, revealing how the Tikhonov parameter influences the decay of filters and the contribution of iterative components.

Contribution

It provides a novel representation of CGT iterates as filtered sums of Lanczos vectors, linking the Tikhonov parameter to filter decay and offering insights for parameter selection.

Findings

Lanczos filters decay faster as the Tikhonov parameter increases.

Filtering out later CG terms improves regularization effectiveness.

Numerical experiments confirm the impact of parameter choices on filtering behavior.

Abstract

We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate stopping rule and CG applied to the system solving for a Tikhonov-regularized solution (CGT) $(A^\ast A + c I_{\mathcal{X}}) x^{(\delta,c)} = A^\ast y^\delta$ are closely related regularization methods that build iterates from the same Krylov subspaces. In this work, we show that the CGT iterate can be expressed as $ x^{(\delta,c)}_m = \sum_{i=1}^{m} \gamma^{(m)}_i(c) z_i^{(m)}v_i, $ where $\left\lbrace\gamma_i^{(m)}(c)\right\rbrace_{i=1}^m$ are functions of the Tikhonov parameter $c$ and $x^{(\delta)}_m = \sum_{i=1}^{m} z_i^{(m)}v_i$ is the $m$-th CG iterate. We call these functions Lanczos filters, and they can be shown to have decay properties as $c\rightarrow\infty$ with the speed of decay increasing with $i$. This has the effect of filtering out the contribution of the later terms of the CG iterate. The filters can be constructed using quantities defined via recursions at each iteration. We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors. The filtration approach also provides a platform for further development of parameter choice rules, and similar representations may hold for other hybrid iterative schemes.