Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

TL;DR

This paper introduces an adaptive stepsize variant of the randomized Kaczmarz method for inconsistent linear systems, providing a geometric interpretation, convergence guarantees, and numerical validation.

Contribution

It proposes a novel adaptive stepsize scheme for the randomized Kaczmarz method with a geometric perspective and proven linear convergence for inconsistent systems.

Findings

Converges linearly in expectation to the least-squares solution

Provides a tight upper bound for convergence rate

Numerical experiments confirm theoretical results

Abstract

We investigate the randomized Kaczmarz method that adaptively updates the stepsize using readily available information for solving inconsistent linear systems. A novel geometric interpretation is provided which shows that the proposed method can be viewed as an orthogonal projection method in some sense. We prove that this method converges linearly in expectation to the unique minimum Euclidean norm least-squares solution of the linear system, and provide a tight upper bound for the convergence of the proposed method. Numerical experiments are also given to illustrate the theoretical results.