Generalized Fractional Repetition Codes for Binary Coded Computations

TL;DR

This paper introduces a numerically stable binary fractional repetition coding method for distributed gradient computation and matrix multiplication, improving stability and flexibility over previous approaches.

Contribution

It presents a new binary coding scheme that is numerically stable, adaptable to heterogeneous servers, and extends to coded matrix multiplication with various trade-offs.

Findings

The proposed binary coding method is numerically stable and avoids complex number operations.

It ensures generator matrix balance for any coded parameters.

The scheme can be extended to coded matrix multiplication with different performance trade-offs.

Abstract

This paper addresses the gradient coding and coded matrix multiplication problems in distributed optimization and coded computing. We present a numerically stable binary coding method which overcomes the drawbacks of the \textit{Fractional Repetition Coding} gradient coding method proposed by Tandon et al., and can also be leveraged by coded computing networks whose servers are of heterogeneous nature. Specifically, we propose a construction for fractional repetition gradient coding; while ensuring that the generator matrix remains close to perfectly balanced for any set of coded parameters, as well as a low complexity decoding step. The proposed binary encoding avoids operations over the real and complex numbers which are inherently numerically unstable, thereby enabling numerically stable distributed encodings of the partial gradients. We then make connections between gradient coding and coded matrix multiplication. Specifically, we show that any gradient coding scheme can be extended to coded matrix multiplication. Furthermore, we show how the proposed binary gradient coding scheme can be used to construct two different coded matrix multiplication schemes, each achieving different trade-offs.