When Do Redundant Requests Reduce Latency ?

TL;DR

This paper provides an analytical framework to determine when redundant requests in distributed systems reduce latency, revealing that their effectiveness depends on service time distributions and removal policies, with optimal strategies varying by load conditions.

Contribution

It introduces a model analyzing the impact of redundancy on latency, characterizing scenarios where redundancy helps or hurts, and guiding optimal redundant-request policies.

Findings

Redundant requests reduce latency with memoryless or heavier service times when jobs can be instantly removed.

Under high load, no redundancy is optimal when service times are lighter or removal is delayed.

Results apply to arbitrary arrival processes, broadening their applicability.

Abstract

Several systems possess the flexibility to serve requests in more than one way. For instance, a distributed storage system storing multiple replicas of the data can serve a request from any of the multiple servers that store the requested data, or a computational task may be performed in a compute-cluster by any one of multiple processors. In such systems, the latency of serving the requests may potentially be reduced by sending "redundant requests": a request may be sent to more servers than needed, and it is deemed served when the requisite number of servers complete service. Such a mechanism trades off the possibility of faster execution of at least one copy of the request with the increase in the delay due to an increased load on the system. Due to this tradeoff, it is unclear when redundant requests may actually help. Several recent works empirically evaluate the latency performance of redundant requests in diverse settings. This work aims at an analytical study of the latency performance of redundant requests, with the primary goals of characterizing under what scenarios sending redundant requests will help (and under what scenarios they will not help), as well as designing optimal redundant-requesting policies. We first present a model that captures the key features of such systems. We show that when service times are i.i.d. memoryless or "heavier", and when the additional copies of already-completed jobs can be removed instantly, redundant requests reduce the average latency. On the other hand, when service times are "lighter" or when service times are memoryless and removal of jobs is not instantaneous, then not having any redundancy in the requests is optimal under high loads. Our results hold for arbitrary arrival processes.