On spectral properties and fast initial convergence of the Kaczmarz method

TL;DR

This paper provides a spectral analysis of the Kaczmarz method, explaining its rapid initial convergence and semi-convergence behavior in solving linear inverse problems, especially in computed tomography.

Contribution

It offers new spectral and statistical insights into the Kaczmarz method's fast initial convergence and semi-convergence, which were previously not well understood.

Findings

Spectral analysis explains rapid initial convergence.

Statistical analysis reveals noise influence on semi-convergence.

Numerical examples validate theoretical insights.

Abstract

The Kaczmarz method is successfully used for solving discretizations of linear inverse problems, especially in computed tomography where it is known as ART. Practitioners often observe and appreciate its fast convergence in the first few iterations, leading to the same favorable semi-convergence that we observe for simultaneous iterative reconstruction methods. While the latter methods have symmetric and positive definite iteration operators that facilitate their analysis, the operator in Kaczmarz's method is nonsymmetric and it has been an open question so far to understand this fast initial convergence. We perform a spectral analysis of Kaczmarz's method that gives new insight into its (often fast) initial behavior. We also carry out a statistical analysis of how the data noise enters the iteration vectors, which sheds new light on the semi-convergence. Our results are illustrated with several numerical examples.