Randomized extended block Kaczmarz for solving least squares

TL;DR

This paper introduces a simple randomized extended block Kaczmarz algorithm that efficiently solves all types of linear systems, converges exponentially, and is suitable for distributed computing, outperforming existing methods.

Contribution

The paper presents a pseudoinverse-free, exponentially convergent randomized extended block Kaczmarz algorithm applicable to all linear systems, with efficient distributed implementation.

Findings

Exponential mean square convergence to least squares solution.

Works for all linear system types, including inconsistent and rank-deficient.

Demonstrates significant computational time improvements in numerical tests.

Abstract

Randomized iterative algorithms have recently been proposed to solve large-scale linear systems. In this paper, we present a simple randomized extended block Kaczmarz algorithm that exponentially converges in the mean square to the unique minimum $\ell_2$-norm least squares solution of a given linear system of equations. The proposed algorithm is pseudoinverse-free and therefore different from the projection-based randomized double block Kaczmarz algorithm of Needell, Zhao, and Zouzias. We emphasize that our method works for all types of linear systems (consistent or inconsistent, overdetermined or underdetermined, full-rank or rank-deficient). Moreover, our approach can utilize efficient implementations on distributed computing units, yielding remarkable improvements in computational time. Numerical examples are given to show the efficiency of the new algorithm.