Straggler-tolerant stationary methods for linear systems

TL;DR

This paper introduces straggler-tolerant iterative methods for solving linear systems that are robust to partial and random matrix-vector computations, applicable in distributed cloud environments with unreliable workers.

Contribution

It proposes novel Richardson and Chebyshev semi-iterative schemes that tolerate stragglers and provides convergence analysis under random partial computations.

Findings

Proved convergence conditions in expectation for the proposed schemes.

Numerical experiments confirm theoretical results and effectiveness on sparse matrices.

Abstract

In this paper, we consider the iterative solution of linear algebraic equations under the condition that matrix-vector products with the coefficient matrix are computed only partially. At the same time, non-computed entries are set to zeros. We assume that both the number of computed entries and their associated row index set are random variables, with the row index set sampled uniformly given the number of computed entries. This model of computations is realized in hybrid cloud computing architectures following the controller-worker distributed model under the influence of straggling workers. We propose straggler-tolerant Richardson iteration scheme and Chebyshev semi-iterative schemes, and prove sufficient conditions for their convergence in expectation. Numerical experiments verify the presented theoretical results as well as the effectiveness of the proposed schemes on a few sparse matrix problems.