Random Reordering in SOR-Type Methods

TL;DR

This paper provides theoretical support for the observed improvement of randomized reordering in SOR-type methods for solving linear systems, showing that randomized versions outperform cyclic ordering asymptotically.

Contribution

It introduces and analyzes shuffled and preshuffled SOR methods, offering new error bounds that surpass previous cyclic ordering bounds.

Findings

Randomized reordering improves SOR performance on average.

Error bounds for randomized SOR versions are asymptotically better.

Analysis based on triangular truncation of Hermitian matrices.

Abstract

When iteratively solving linear systems By=b with Hermitian positive semi-definite $B$, and in particular when solving least-squares problems for $Ax=b$ by reformulating them as $AA^\ast y=b$, it is often observed that SOR-type methods (Gauss-Seidel, Kaczmarz) perform suboptimally for the given equation ordering, and that random reordering improves the situation on average.This paper is an attempt to provide some additional theoretical support for this phenomenon. We show eerror bounds for two randomized versions, called shuffled and preshuffled SOR, that improve asymptotically upon the best known bounds fro SOR with cyclic ordering. Our results are based on studying the behavior of the triangular truncation of Hermitian matrices with respect to their permutations.