A CG-type method in Banach spaces with an application to computerized tomography

TL;DR

This paper extends conjugate gradient methods to Banach spaces using a subspace optimization approach, demonstrating improved convergence in applications like computerized tomography, especially in lp-spaces with small p.

Contribution

It introduces a novel SESOP-based method that generalizes CG to Banach spaces, employing metric projections for orthogonalization, and shows its effectiveness through theoretical analysis and numerical experiments.

Findings

The method coincides with Polak-Ribière CG in l2-space.

It exhibits weak convergence in Banach spaces.

Numerical results show superior convergence in lp-spaces, especially with small p.

Abstract

Conjugate Gradient (CG) methods are one of the most effective iterative methods to solve linear equations in Hilbert spaces. So far, they have been inherently bound to these spaces since they make use of the inner product structure. In more general Banach spaces one of the most prominent iterative solvers are Landweber-type methods that essentially resemble the Steepest Descent method applied to the normal equation. More advanced are subspace methods that take up the idea of a Krylov-type search space, wherein an optimal solution is sought. However, they do not share the conjugacy property with CG methods. In this article we propose that the Sequential Subspace Optimization (SESOP) method can be considered as an extension of CG methods to Banach spaces. We employ metric projections to orthogonalize the current search direction with respect to the search space from the last iteration. For the l2-space our method then exactly coincides with the Polak-Ribi\`ere type of the CG method when applied to the normal equation. We show that such an orthogonalized search space still leads to weak convergence of the subspace method. Moreover, numerical experiments on a random matrix toy problem and 2D computerized tomography on lp-spaces show superior convergence properties over all p compared to non-orthogonalized search spaces. This especially holds for lp-spaces with small p. We see that the closer we are to an l2-space, the more we recover of the conjugacy property that holds in these spaces, i. e., as expected, the more the convergence behaves independently of the size of the truncated search space.