All-to-All Encode in Synchronous Systems

TL;DR

This paper introduces the all-to-all encode operation in synchronous systems, providing a universal, optimal algorithm for linear combinations of data packets with optimized communication efficiency for specific code types.

Contribution

It defines all-to-all encode, develops a universal algorithm for it, and proves its optimality, with optimized variants for Vandermonde and Lagrange codes.

Findings

Universal algorithm computes any matrix in the model.

Proven optimality of the algorithm.

Optimized communication for Vandermonde and Lagrange codes.

Abstract

We define all-to-all encode, a collective communication operation serving as a primitive in decentralized computation and storage systems. Consider a scenario where every processor initially has a data packet and requires a linear combination of all data packets; the linear combinations are distinct from one processor to another, and are specified by a generator matrix of an error correcting code. We use a linear network model, in which processors transmit linear combinations of their data and previously received packets, and adopt a standard synchronous system setting to analyze its communication cost. We provide a universal algorithm which computes any matrix in this model by only varying intermediate coefficients, and prove its optimality. When the generator matrix is of the Vandermonde or Lagrange type, we further optimize the communication efficiency of the proposed algorithm.