A Storage-Computation-Communication Tradeoff for Distributed Computing

TL;DR

This paper introduces a new coded scheme for distributed computing that optimally balances storage, computation, and communication, reducing data transfer while maintaining flexibility in system design.

Contribution

It proposes the D3C scheme, achieving minimal communication load for given storage and enabling flexible tradeoffs between storage and computation.

Findings

D3C achieves the lowest possible communication load for a given storage.

D3C requires fewer intermediate values to be computed compared to previous schemes.

The scheme allows flexible tradeoffs between storage space and computation load.

Abstract

This paper investigates distributed computing systems where computations are split into "Map" and "Reduce" functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade between storage space and the number of IVAs to be computed. Communication load is then analyzed for any given tradeoff.