Convergence rates for Kaczmarz-type algorithms

TL;DR

This paper provides a theoretical analysis of convergence rates for various Kaczmarz algorithms, establishing linear and sublinear rates for different control strategies, thereby advancing understanding of their efficiency.

Contribution

It introduces new convergence rate results for Kaczmarz algorithms with specific control sequences, including almost cyclic and remotest set strategies, complementing existing random selection analyses.

Findings

Proves at least linear convergence for Kaczmarz-Tanabe and extended methods.

Establishes sublinear convergence for almost cyclic control strategy.

Demonstrates linear convergence for remotest set control strategy.

Abstract

In this paper we make a theoretical analysis of the convergence rates of Kaczmarz and Extended Kaczmarz projection algorithms for some of the most practically used control sequences. We first prove an at least linear convergence rate for the Kaczmarz-Tanabe and its Extended version methods (the one in which a complete set of projections using row/column index is performed in each iteration). Then we apply the main ideas of this analysis in establishing an at least sublinear, respectively linear convergence rate for the Kaczmarz algorithm with almost cyclic and the remotest set control strategies, and their extended versions, respectively. These results complete the existing ones related to the random selection procedures.