Coded Distributed Computing with Heterogeneous Function Assignments

TL;DR

This paper extends coded distributed computing (CDC) to heterogeneous networks by proposing new schemes that assign functions variably across nodes, reducing communication load compared to traditional homogeneous CDC.

Contribution

It introduces two novel heterogeneous CDC approaches, including a scheme with variable function assignment and an expandable design for improved efficiency.

Findings

Heterogeneous CDC schemes can outperform homogeneous ones in communication load.

The proposed expandable scheme serves multiple nodes simultaneously in the Shuffle phase.

Heterogeneous function assignment reduces communication costs in certain scenarios.

Abstract

Coded distributed computing (CDC) introduced by Li et. al. is an effective technique to trade computation load for communication load in a MapReduce framework. CDC achieves an optimal trade-off by duplicating map computations at $r$ computing nodes to yield multicasting opportunities such that $r$ nodes are served simultaneously in the Shuffle phase. However, in general, the state-of-the-art CDC scheme is mainly designed only for homogeneous networks, where the computing nodes are assumed to have the same storage, computation and communication capabilities. In this work, we explore two novel approaches of heterogeneous CDC design. First, we study CDC schemes which operate on multiple, collaborating homogeneous computing networks. Second, we allow heterogeneous function assignment in the CDC design, where nodes are assigned a varying number of reduce functions. Finally, we propose an expandable heterogeneous CDC scheme where $r-1$ nodes are served simultaneously in the Shuffle phase. In comparison to the state-of-the-art homogeneous CDC scheme with an equivalent computation load, we find our newly proposed heterogeneous CDC scheme has a smaller communication load in some cases.