Straggler-Aware Coded Polynomial Aggregation

TL;DR

This paper extends coded polynomial aggregation to handle stragglers in distributed systems, providing conditions for exact recovery based on non-straggler patterns and demonstrating improved efficiency over traditional methods.

Contribution

It introduces a straggler-aware CPA scheme with necessary and sufficient conditions for exact recovery based on non-straggler pattern intersections.

Findings

Exact recovery achievable with fewer responses than polynomial codes.

Feasibility characterized by intersection structure of non-straggler sets.

Simulation confirms sharp transition at intersection threshold.

Abstract

Coded polynomial aggregation (CPA) in distributed computing systems enables the master to directly recover a weighted aggregation of polynomial computations without individually decoding each term, thereby reducing the number of required worker responses. However, existing CPA schemes are restricted to an idealized setting in which the system cannot tolerate stragglers. In this paper, we extend CPA to straggler-aware distributed computing systems with a pre-specified non-straggler pattern, where exact recovery is required for a given collection of admissible non-straggler sets. Our main results show that exact recovery of the desired aggregation is achievable with fewer worker responses than that required by polynomial codes based on individual decoding, and that feasibility is characterized by the intersection structure of the non-straggler patterns. In particular, we establish necessary and sufficient conditions for exact recovery in straggler-aware CPA. We identify an intersection-size threshold that is sufficient to guarantee exact recovery. When the number of admissible non-straggler sets is sufficiently large, we further show that this threshold is necessary in a generic sense. We also provide an explicit construction of feasible CPA schemes whenever the intersection size exceeds the derived threshold. Finally, simulations verify our theoretical results by demonstrating a sharp feasibility transition at the predicted intersection threshold.