On the Polyak momentum variants of the greedy deterministic single and multiple row-action methods

TL;DR

This paper introduces momentum-accelerated greedy deterministic row-action methods for solving linear systems, demonstrating linear convergence to minimum-norm solutions and validating results through numerical experiments and real-world applications.

Contribution

It develops novel Polyak momentum variants of the greedy Kaczmarz method with proven convergence properties and practical effectiveness.

Findings

Algorithms converge linearly to least-squares solutions.

Numerical experiments confirm theoretical convergence rates.

Applications in data fitting demonstrate practical utility.

Abstract

For solving a consistent system of linear equations, the classical row-action (also known as Kaczmarz) method is a simple while really effective iteration solver. Based on the greedy index selection strategy and Polyak's heavy-ball momentum acceleration technique, we propose two deterministic row-action methods and establish the corresponding convergence theory. We show that our algorithm can linearly converge to a least-squares solution with minimum Euclidean norm. Several numerical studies have been presented to corroborate our theoretical findings. Real-world applications, such as data fitting in computer-aided geometry design, are also presented for illustrative purposes.