On greedy multi-step inertial randomized Kaczmarz method for solving linear systems

TL;DR

This paper introduces a greedy multi-step inertial randomized Kaczmarz (GMIRK) method that improves convergence rates for solving large linear systems by incorporating a greedy probability criterion and analyzing its geometric and theoretical properties.

Contribution

The paper proposes a novel GMIRK method with enhanced convergence guarantees and explores its geometric interpretation and relation to existing projection methods.

Findings

GMIRK achieves faster convergence than previous methods.

The method's geometric interpretation links it to orthogonal and oblique projections.

Numerical experiments confirm the theoretical improvements.

Abstract

The multi-step inertial randomized Kaczmarz (MIRK) method is an iterative method for solving large-scale linear systems. In this paper, we enhance the MIRK method by incorporating the greedy probability criterion, coupled with the introduction of a tighter threshold parameter for this criterion. We prove that the proposed greedy MIRK (GMIRK) method enjoys an improved deterministic linear convergence compared to both the MIRK method and the greedy randomized Kaczmarz method. Furthermore, we exhibit that the multi-step inertial extrapolation approach can be geometrically interpreted as an orthogonal projection method, and establish its relationship with the sketch-and-project method in (SIAM J. Matrix Anal. Appl. 36(4):1660-1690, 2015) and the oblique projection technique in (Results Appl. Math. 16:100342, 2022). Numerical experiments are provided to confirm our results.