On the Convergence of Randomized and Greedy Relaxation Schemes for Solving Nonsingular Linear Systems of Equations

TL;DR

This paper extends convergence results for randomized and greedy relaxation methods to certain non-Hermitian matrices, providing bounds that match those of more costly deterministic algorithms, supported by numerical experiments.

Contribution

It generalizes convergence analysis to non-Hermitian matrices and identifies optimal parameter choices for randomized and greedy relaxation schemes.

Findings

Convergence bounds for randomized methods are as good as deterministic algorithms.

Optimal parameter choices improve convergence bounds.

Numerical experiments confirm theoretical results.

Abstract

We extend results known for the randomized Gauss-Seidel and the Gauss-Southwell methods for the case of a Hermitian and positive definite matrix to certain classes of non-Hermitian matrices. We obtain convergence results for a whole range of parameters describing the probabilities in the randomized method or the greedy choice strategy in the Gauss-Southwell-type methods. We identify those choices which make our convergence bounds best possible. Our main tool is to use weighted l1-norms to measure the residuals. A major result is that the best convergence bounds that we obtain for the expected values in the randomized algorithm are as good as the best for the deterministic, but more costly algorithms of Gauss-Southwell type. Numerical experiments illustrate the convergence of the method and the bounds obtained. Comparisons with the randomized Kaczmarz method are also presented.