Randomized Extended Kaczmarz for Solving Least-Squares

TL;DR

This paper introduces a randomized iterative algorithm that efficiently converges to the least squares solution of linear systems, with performance depending on the system's condition number and sparsity.

Contribution

It extends the randomized Kaczmarz method to achieve exponential convergence for least squares problems, providing theoretical complexity bounds.

Findings

Exponential convergence in expectation to the least squares solution.

Expected computational complexity depends on the system's condition number and sparsity.

Extension of the randomized Kaczmarz method with proven convergence properties.

Abstract

We present a randomized iterative algorithm that exponentially converges in expectation to the minimum Euclidean norm least squares solution of a given linear system of equations. The expected number of arithmetic operations required to obtain an estimate of given accuracy is proportional to the square condition number of the system multiplied by the number of non-zeros entries of the input matrix. The proposed algorithm is an extension of the randomized Kaczmarz method that was analyzed by Strohmer and Vershynin.