Successive Approximation Coding for Distributed Matrix Multiplication

TL;DR

This paper introduces successive approximation coding (SAC) for distributed matrix multiplication, enabling a tradeoff between accuracy and speed, and progressively improving results as more nodes complete their tasks.

Contribution

It proposes SAC techniques that combine approximate and coded computing, providing a new method for faster, progressively accurate distributed matrix multiplication.

Findings

SAC achieves a better accuracy-speed tradeoff than previous methods.

SAC can exactly recover the desired computation with enough compute nodes.

Theoretical guidelines for SAC design are provided.

Abstract

Coded distributed computing was recently introduced to mitigate the effect of stragglers on distributed computing. This paper combines ideas of approximate computing with coded computing to further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy and speed, allowing the distributed computing system to produce approximations that increase in accuracy over time. If a sufficient number of compute nodes finish their tasks, SAC exactly recovers the desired computation. We theoretically provide design guidelines for our SAC techniques, and numerically show that SAC achieves a better accuracy-speed tradeoff in comparison with previous methods.