A New Combinatorial Coded Design for Heterogeneous Distributed Computing

TL;DR

This paper introduces a combinatorial coded design for heterogeneous distributed computing that reduces data requirements and maintains high efficiency, overcoming limitations of previous homogeneous CDC schemes.

Contribution

It proposes a novel combinatorial CDC approach tailored for heterogeneous networks, significantly reducing data complexity while preserving optimal trade-offs.

Findings

Reduces input file requirements exponentially compared to original CDC.

Maintains multiplicative computation-communication gain.

Achieves asymptotic optimal performance in heterogeneous settings.

Abstract

Coded Distributed Computing (CDC) introduced by Li et al. in 2015 offers an efficient approach to trade computing power to reduce the communication load in general distributed computing frameworks such as MapReduce and Spark. In particular, increasing the computation load in the Map phase by a factor of r can create coded multicasting opportunities to reduce the communication load in the Shuffle phase by the same factor. However, the CDC scheme is designed for the homogeneous settings, where the storage, computation load and communication load on the computing nodes are the same. In addition, it requires an exponentially large number of input files (data batches), reduce functions and multicasting groups relative to the number of nodes to achieve the promised gain. We address the CDC limitations by proposing a novel CDC approach based on a combinatorial design, which accommodates heterogeneous networks where nodes have varying storage and computing capabilities. In addition, the proposed approach requires an exponentially less number of input files compared to the original CDC scheme proposed by Li et al. Meanwhile, the resulting computation-communication trade-off maintains the multiplicative gain compared to conventional uncoded unicast and asymptotically achieves the optimal performance proposed by Li et al.