Threshold-based rerouting and replication for resolving job-server affinity relations

TL;DR

This paper investigates threshold-based rerouting and replication policies in a multi-server system with unknown or partially known job types, analyzing their impact on stability and latency to optimize job processing.

Contribution

It introduces a framework for analyzing rerouting and replication policies considering job type uncertainty and proposes threshold-based policies to improve latency.

Findings

Replication maximizes stability with unbalanced service speeds.

Rerouting can improve latency for partly known job types.

Uncertainty in job types can significantly reduce system performance.

Abstract

We consider a system with several job types and two parallel server pools. Within the pools the servers are homogeneous, but across pools possibly not in the sense that the service speed of a job may depend on its type as well as the server pool. Immediately upon arrival, jobs are assigned to a server pool. This could be based on (partial) knowledge of their type, but such knowledge might not be available. Information about the job type can however be obtained while the job is in service; as the service progresses, the likelihood that the service speed of this job type is low increases, creating an incentive to execute the job on different, possibly faster, server(s). Two policies are considered: reroute the job to the other server pool, or replicate it there. We determine the effective load per server under both the rerouting and replication policy for completely unknown as well as partly known job types. We also examine the impact of these policies on the stability bound, and find that the uncertainty in job types may significantly degrade the performance. For (highly) unbalanced service speeds full replication achieves the largest stability bound while for (nearly) balanced service speeds no replication maximizes the stability bound. Finally, we discuss how the use of threshold-based policies can help improve the expected latency for completely or partly unknown job types.