Efficient Task Replication for Fast Response Times in Parallel Computation

TL;DR

This paper develops a theoretical framework to analyze the trade-off between response time and resource usage in large-scale distributed computing, proposing algorithms for optimal task replication strategies.

Contribution

It introduces a novel analytical model for task replication, identifying conditions where replication improves both response time and resource efficiency.

Findings

Replication can reduce both response time and resource usage under certain conditions.

The paper provides algorithms for finding optimal or near-optimal replication policies.

Insights guide scheduler design in large-scale distributed systems.

Abstract

One typical use case of large-scale distributed computing in data centers is to decompose a computation job into many independent tasks and run them in parallel on different machines, sometimes known as the "embarrassingly parallel" computation. For this type of computation, one challenge is that the time to execute a task for each machine is inherently variable, and the overall response time is constrained by the execution time of the slowest machine. To address this issue, system designers introduce task replication, which sends the same task to multiple machines, and obtains result from the machine that finishes first. While task replication reduces response time, it usually increases resource usage. In this work, we propose a theoretical framework to analyze the trade-off between response time and resource usage. We show that, while in general, there is a tension between response time and resource usage, there exist scenarios where replicating tasks judiciously reduces completion time and resource usage simultaneously. Given the execution time distribution for machines, we investigate the conditions for a scheduling policy to achieve optimal performance trade-off, and propose efficient algorithms to search for optimal or near-optimal scheduling policies. Our analysis gives insights on when and why replication helps, which can be used to guide scheduler design in large-scale distributed computing systems.