Reflective block Kaczmarz algorithms for least squares

TL;DR

This paper studies and extends reflective block Kaczmarz algorithms for least squares problems, providing theoretical convergence analysis, block versions, and numerical verification of their efficiency.

Contribution

It offers a comprehensive analysis and extension of two recent Kaczmarz algorithms, including block versions and convergence rates, with numerical validation.

Findings

Theoretical convergence rates for the algorithms.

Effective block versions of the algorithms.

Numerical experiments confirm efficiency.

Abstract

In [Steinerberger, Q. Appl. Math., 79:3, 419-429, 2021] and [Shao, SIAM J. Matrix Anal. Appl. 44(1), 212-239, 2023], two new types of Kaczmarz algorithms, which share some similarities, for consistent linear systems were proposed. These two algorithms not only compete with many previous Kaczmarz algorithms but, more importantly, reveal some interesting new geometric properties of solutions to linear systems that are not obvious from the standard viewpoint of the Kaczmarz algorithm. In this paper, we comprehensively study these two algorithms. First, we theoretically analyse the algorithms given in [Steinerberger, Q. Appl. Math., 79:3, 419-429, 2021] for solving least squares. Second, we extend the two algorithms to block versions and provide their theoretical convergence rates. Our numerical experiments also verify the efficiency of these algorithms. Third, as a theoretical complement, we address some key questions left unanswered in [Shao, SIAM J. Matrix Anal. Appl. 44(1), 212-239, 2023].