A Note On The Randomized Kaczmarz Method With A Partially Weighted Selection Step

TL;DR

This paper introduces a new partially weighted selection scheme for the randomized Kaczmarz method, aiming to reduce residual computations while maintaining convergence speed, supported by numerical evidence.

Contribution

It proposes an alternative random selection scheme that integrates into the classical Kaczmarz algorithm, reducing residual calculations without sacrificing convergence.

Findings

Residual computations can be significantly reduced.

The new scheme maintains convergence speed.

Numerical examples confirm moderate residual requirements.

Abstract

In this note we reconsider two known algorithms which both usually converge faster than the randomized Kaczmarz method introduced by Strohmer and Vershynin(2009), but require the additional computation of all residuals of an iteration at each step. As already indicated in the literature, e.g. arXiv:2007.02910 and arXiv:2011.14693, it is shown that the non-randomized version of the two algorithms converges at least as fast as the randomized version, while still requiring computation of all residuals. Based on that observation, a new simple random sample selection scheme has been introduced by arXiv:2011.14693 to reduce the required total of residuals. In the same light we propose an alternative random selection scheme which can easily be included as a `partially weighted selection step' into the classical randomized Kaczmarz algorithm without much ado. Numerical examples show that the randomly determined number of required residuals can be quite moderate.