Distributed Linearly Separable Computation

TL;DR

This paper investigates the optimal tradeoff between computation and communication costs in distributed linear separable function computation, proposing schemes and bounds that generalize existing distributed coding problems.

Contribution

It introduces novel achievability schemes and converse bounds for minimal computation cost scenarios, extending distributed gradient coding to recover multiple linear combinations.

Findings

Achievability and converse bounds match for certain parameters.

Proposed schemes are optimal under cyclic dataset assignment.

Enables recovery of multiple linear combinations beyond the gradient coding setting.

Abstract

This paper formulates a distributed computation problem, where a master asks $N$ distributed workers to compute a linearly separable function. The task function can be expressed as $K_c$ linear combinations of $K$ messages, where each message is a function of one dataset. Our objective is to find the optimal tradeoff between the computation cost (number of uncoded datasets assigned to each worker) and the communication cost (number of symbols the master must download), such that from the answers of any $N_r$ out of $N$ workers the master can recover the task function with high probability, where the coefficients of the $K_c$ linear combinations are uniformly i.i.d. over some large enough finite field. The formulated problem can be seen as a generalized version of some existing problems, such as distributed gradient coding and distributed linear transform. In this paper, we consider the specific case where the computation cost is minimum, and propose novel achievability schemes and converse bounds for the optimal communication cost. Achievability and converse bounds coincide for some system parameters; when they do not match, we prove that the achievable distributed computing scheme is optimal under the constraint of a widely used `cyclic assignment' scheme on the datasets. Our results also show that when $K = N$, with the same communication cost as the optimal distributed gradient coding scheme proposed by Tandon et al. from which the master recovers one linear combination of $K$ messages, our proposed scheme can let the master recover any additional $N_r - 1$ independent linear combinations of messages with high probability.