Two-subspace Projection Method for Coherent Overdetermined Systems

TL;DR

This paper introduces a two-subspace projection extension of the Kaczmarz method for solving overdetermined linear systems, demonstrating faster convergence especially with correlated data.

Contribution

It proposes a novel two-row projection algorithm that enhances convergence rates over the standard Kaczmarz method for certain systems.

Findings

Exponential convergence in expectation for the proposed method.

Improved convergence rate over the randomized Kaczmarz method with correlated rows.

Experimental validation showing significant performance gains.

Abstract

We present a Projection onto Convex Sets (POCS) type algorithm for solving systems of linear equations. POCS methods have found many applications ranging from computer tomography to digital signal and image processing. The Kaczmarz method is one of the most popular solvers for overdetermined systems of linear equations due to its speed and simplicity. Here we introduce and analyze an extension of the Kaczmarz method that iteratively projects the estimate onto a solution space given by two randomly selected rows. We show that this projection algorithm provides exponential convergence to the solution in expectation. The convergence rate improves upon that of the standard randomized Kaczmarz method when the system has correlated rows. Experimental results confirm that in this case our method significantly outperforms the randomized Kaczmarz method.