Block sampling Kaczmarz-Motzkin methods for consistent linear systems

TL;DR

This paper introduces two block sampling Kaczmarz-Motzkin methods for solving consistent linear systems, which improve convergence speed and efficiency over the existing SKM method through greedy strategies and simultaneous constraint enforcement.

Contribution

The paper proposes novel block sampling Kaczmarz-Motzkin algorithms with greedy strategies, providing theoretical convergence guarantees and demonstrating superior performance over SKM.

Findings

Block methods converge at least as fast as SKM.

Numerical experiments show fewer iterations and less computing time.

Methods outperform SKM in accuracy and efficiency.

Abstract

The sampling Kaczmarz-Motzkin (SKM) method is a generalization of the randomized Kaczmarz and Motzkin methods. It first samples some rows of coefficient matrix randomly to build a set and then makes use of the maximum violation criterion within this set to determine a constraint. Finally, it makes progress by enforcing this single constraint. In this paper, on the basis of the framework of the SKM method and considering the greedy strategies, we present two block sampling Kaczmarz-Motzkin methods for consistent linear systems. Specifically, we also first sample a subset of rows of coefficient matrix and then determine an index in this set using the maximum violation criterion. Unlike the SKM method, in the rest of the block methods, we devise different greedy strategies to build index sets. Then, the new methods make progress by enforcing the corresponding multiple constraints simultaneously. Theoretical analyses demonstrate that these block methods converge at least as quickly as the SKM method, and numerical experiments show that, for the same accuracy, our methods outperform the SKM method in terms of the number of iterations and computing time.