Approximate Distributed Coded Computing: Polynomial Codes and Randomized Sketching

TL;DR

This paper reviews and combines coded computing and randomized numerical linear algebra techniques to develop distributed schemes that accelerate optimization and machine learning tasks despite slow or unresponsive servers.

Contribution

It introduces novel distributed schemes that integrate polynomial codes and randomized sketching, enhancing robustness and efficiency in large-scale distributed computations.

Findings

Combined coding and randomized methods improve system resilience.

Distributed schemes accelerate machine learning algorithms.

The approach addresses server latency and non-responsiveness.

Abstract

Coded computing is a distributed paradigm that uses coding theory to introduce \textit{redundancy} and overcome bottlenecks in large-scale systems. In the same vein, randomized numerical linear algebra employs probabilistic methods to \textit{compress} and accelerate linear algebraic operations, addressing challenges in high-dimensional data analysis. This article reviews the foundations of both fields and presents distributed schemes that combine techniques from both to speed up optimization and machine learning algorithms, in the presence of slow or non-responsive servers. Along the way, we touch on various related topics and mathematical concepts.