Augmented Flexible Krylov Subspace methods with applications to Bayesian inverse problems

TL;DR

This paper introduces two new augmented flexible Krylov subspace methods, AF-GMRES and AF-LSQR, for efficiently solving large-scale Bayesian inverse problems with combined smooth and sparse regularization, offering convergence guarantees and improved stability.

Contribution

The paper develops novel augmented flexible Krylov methods that incorporate multiple regularization terms into a single subspace, enabling on-the-fly parameter selection and enhanced stability for large-scale inverse problems.

Findings

Methods demonstrate improved convergence and stability.

Effective in synthetic and real-world inverse problems.

Outperform traditional reweighted norm schemes.

Abstract

This paper presents two new augmented flexible (AF)-Krylov subspace methods, AF-GMRES and AF-LSQR, to compute solutions of large-scale linear discrete ill-posed problems that can be modeled as the sum of two independent random variables, exhibiting smooth and sparse stochastic characteristics respectively. Following a Bayesian modelling approach, this corresponds to adding a covariance-weighted quadratic term and a sparsity enforcing $\ell_1$ term in the original least-squares minimization scheme. To handle the $\ell_1$ regularization term, the proposed approach constructs a sequence approximating quadratic problems that are partially solved using augmented flexible Krylov-Tikhonov methods. Compared to other traditional methods used to solve this minimization problem, such as those based on iteratively reweighted norm schemes, the new algorithms build a single (augmented, flexible) approximation (Krylov) subspace that encodes information about the different regularization terms through adaptable "preconditioning". The solution space is then expanded as soon as a new problem within the sequence is defined. This also allows for the regularization parameters to be chosen on-the-fly at each iteration. Compared to most recent work on generalized flexible Krylov methods, our methods offer theoretical assurance of convergence and a more stable numerical performance. The efficiency of the new methods is shown through a variety of experiments, including a synthetic image deblurring problem, a synthetic atmospheric transport problem, and fluorescence molecular tomography reconstructions using both synthetic and real-world experimental data.