Semi-randomized block Kaczmarz methods with simple random sampling for large-scale linear systems

TL;DR

This paper introduces semi-randomized block Kaczmarz methods that efficiently solve large-scale linear systems by avoiding full data scans, using simple random sampling, and enabling simultaneous updates for multiple right-hand sides.

Contribution

It proposes novel semi-randomized block Kaczmarz algorithms that reduce data access and computational costs, suitable for large-scale and multi-right-hand-side systems.

Findings

Methods outperform state-of-the-art algorithms on real-world data

No need to scan all matrix rows or compute residuals each iteration

Effective for large-scale systems with multiple right-hand sides

Abstract

Randomized block Kaczmraz method plays an important role in solving large-scale linear system. One of the key points of this type of methods is how to effectively select working rows. However, in most of the state-of-the-art randomized block Kaczmarz-type methods, one has to scan all the rows of the coefficient matrix in advance for computing probabilities or paving, or to compute the residual vector of the linear system in each iteration to determine the working rows. Thus, we have to access all the rows of the data matrix in these methods, which are unfavorable for big-data problems. Moreover, to the best of our knowledge, how to efficiently choose working rows in randomized block Kaczmarz-type methods for multiple linear systems is still an open problem. In order to deal with these problems, we propose semi-randomized block Kaczmarz methods with simple random sampling for linear systems with single and multiple right-hand sides, respectively. In these methods, there is no need to scan or pave all the rows of the coefficient matrix, nor to compute probabilities and the residual vector of the linear system in each step. Specifically, one can update all the solutions of a large-scale linear system with multiple right-hand sides simultaneously. The convergence of the proposed methods is considered. Numerical experiments on both real-world and synthetic data sets show that the proposed methods are superior to many state-of-the-art randomized Kaczmarz-type methods for large-scale linear systems.