On Heterogeneous Coded Distributed Computing
Mehrdad Kiamari, Chenwei Wang, and A. Salman Avestimehr

TL;DR
This paper extends the Coded Distributed Computing framework to heterogeneous systems, optimizing data placement and communication to reduce load, with proven optimal schemes for 3-node clusters and algorithms for larger networks.
Contribution
It introduces the first information-theoretic optimal data placement and coding scheme for heterogeneous CDC systems, generalizing to larger networks.
Findings
Optimal data placement for 3-node clusters
Coded shuffling scheme minimizes communication load
Algorithmic generalization for larger networks
Abstract
We consider the recently proposed Coded Distributed Computing (CDC) framework that leverages carefully designed redundant computations to enable coding opportunities that substantially reduce the communication load of distributed computing. We generalize this framework to heterogeneous systems where different nodes in the computing cluster can have different storage (or processing) capabilities. We provide the information-theoretically optimal data set placement and coded data shuffling scheme that minimizes the communication load in a cluster with 3 nodes. For clusters with nodes, we provide an algorithm description to generalize our coding ideas to larger networks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
