Stream Distributed Coded Computing

TL;DR

This paper introduces a novel framework for distributed coded computing that optimally schedules and assigns tasks to heterogeneous workers in streaming environments, significantly reducing job delays compared to naive methods.

Contribution

It presents the first end-to-end joint scheduling and coding framework for heterogeneous distributed streaming computations, optimizing load distribution based on worker capabilities.

Findings

Performance is dramatically better than naive uniform load splitting.

Framework achieves near-ideal job execution delay.

Effective in heterogeneous, streaming computational environments.

Abstract

The emerging large-scale and data-hungry algorithms require the computations to be delegated from a central server to several worker nodes. One major challenge in the distributed computations is to tackle delays and failures caused by the stragglers. To address this challenge, introducing efficient amount of redundant computations via distributed coded computation has received significant attention. Recent approaches in this area have mainly focused on introducing minimum computational redundancies to tolerate certain number of stragglers. To the best of our knowledge, the current literature lacks a unified end-to-end design in a heterogeneous setting where the workers can vary in their computation and communication capabilities. The contribution of this paper is to devise a novel framework for joint scheduling-coding, in a setting where the workers and the arrival of stream computational jobs are based on stochastic models. In our initial joint scheme, we propose a systematic framework that illustrates how to select a set of workers and how to split the computational load among the selected workers based on their differences in order to minimize the average in-order job execution delay. Through simulations, we demonstrate that the performance of our framework is dramatically better than the performance of naive method that splits the computational load uniformly among the workers, and it is close to the ideal performance.