Two-dimensional greedy randomized extended Kaczmarz methods

TL;DR

This paper introduces two novel two-dimensional greedy randomized extended Kaczmarz methods for large linear least-squares problems, demonstrating improved convergence and computational efficiency over existing methods, especially for big data applications.

Contribution

The paper proposes new greedy and semi-randomized Kaczmarz algorithms that select multiple entries per iteration, enhancing convergence and efficiency for large-scale least-squares problems.

Findings

The proposed methods outperform existing randomized Kaczmarz methods in convergence speed.

Numerical experiments show significant reduction in computing time.

Methods are particularly effective for big data problems.

Abstract

The randomized extended Kaczmarz method, proposed by Zouzias and Freris (SIAM J. Matrix Anal. Appl. 34: 773-793, 2013), is appealing for solving least-squares problems. However, its randomly selecting rows and columns of A with probability proportional to their squared norm is unattractive compared to the greedy strategy. In this paper, we first consider a novel two-dimensional greedy randomized extended Kaczmarz method for solving large linear least-squares problems. The proposed method randomly selects two rows and two columns of A by grasping two larger entries in the magnitude of the corresponding residual vector per iteration. To improve its convergence, we then propose a two-dimensional semi-randomized extended Kaczmarz method and its modified version with simple random sampling, which is particularly favorable for big data problems. The convergence analysis of which is also established. Numerical results on some practical applications illustrate the superiority of the proposed methods compared with state-of-the-art randomized extended Kaczmarz methods, especially in terms of computing time.