On Heterogeneous Coded Distributed Computing

TL;DR

This paper extends the Coded Distributed Computing framework to heterogeneous systems, optimizing data placement and communication to reduce load, with proven optimal schemes for 3-node clusters and algorithms for larger networks.

Contribution

It introduces the first information-theoretic optimal data placement and coding scheme for heterogeneous CDC systems, generalizing to larger networks.

Findings

Optimal data placement for 3-node clusters

Coded shuffling scheme minimizes communication load

Algorithmic generalization for larger networks

Abstract

We consider the recently proposed Coded Distributed Computing (CDC) framework that leverages carefully designed redundant computations to enable coding opportunities that substantially reduce the communication load of distributed computing. We generalize this framework to heterogeneous systems where different nodes in the computing cluster can have different storage (or processing) capabilities. We provide the information-theoretically optimal data set placement and coded data shuffling scheme that minimizes the communication load in a cluster with 3 nodes. For clusters with $K>3$ nodes, we provide an algorithm description to generalize our coding ideas to larger networks.