Technical Report: Dealing with Undependable Workers in Decentralized Network Supercomputing

TL;DR

This paper introduces a new randomized algorithm for large-scale internet supercomputing that effectively handles unreliable and crashing processors, ensuring high-probability task completion despite adversarial faults.

Contribution

The paper presents a novel algorithm that tolerates stronger adversaries in decentralized network supercomputing, with proven efficiency bounds under different fault models.

Findings

Algorithm achieves high probability task completion with bounded rounds.

Efficiency bounds depend on the adversary model, with polynomial and poly-logarithmic constraints.

The approach improves robustness against undependable processors in large distributed systems.

Abstract

Internet supercomputing is an approach to solving partitionable, computation-intensive problems by harnessing the power of a vast number of interconnected computers. This paper presents a new algorithm for the problem of using network supercomputing to perform a large collection of independent tasks, while dealing with undependable processors. The adversary may cause the processors to return bogus results for tasks with certain probabilities, and may cause a subset $F$ of the initial set of processors $P$ to crash. The adversary is constrained in two ways. First, for the set of non-crashed processors $P-F$, the \emph{average} probability of a processor returning a bogus result is inferior to $\frac{1}{2}$. Second, the adversary may crash a subset of processors $F$, provided the size of $P-F$ is bounded from below. We consider two models: the first bounds the size of $P-F$ by a fractional polynomial, the second bounds this size by a poly-logarithm. Both models yield adversaries that are much stronger than previously studied. Our randomized synchronous algorithm is formulated for $n$ processors and $t$ tasks, with $n\le t$, where depending on the number of crashes each live processor is able to terminate dynamically with the knowledge that the problem is solved with high probability. For the adversary constrained by a fractional polynomial, the round complexity of the algorithm is $O(\frac{t}{n^\varepsilon}\log{n}\log{\log{n}})$, its work is $O(t\log{n} \log{\log{n}})$ and message complexity is $O(n\log{n}\log{\log{n}})$. For the poly-log constrained adversary, the round complexity is $O(t)$, work is $O(t n^{\varepsilon})$, %$O(t \, poly \log{n})$, and message complexity is $O(n^{1+\varepsilon})$ %$O(n \, poly \log{n})$. All bounds are shown to hold with high probability.