Load balancing policies without feedback using timed replicas

TL;DR

This paper introduces novel load balancing policies without server feedback, using timed replicas and timers to improve performance in certain regimes, simplifying implementation compared to feedback-based methods.

Contribution

It proposes simple, feedback-free load balancing policies using timers and replicas, demonstrating their effectiveness in specific operating regimes.

Findings

Policies outperform some feedback-based policies in certain regimes

No server feedback or replica cancellation needed for effective load balancing

Simple implementation with timers and replicas

Abstract

Dispatching policies such as the join shortest queue (JSQ), join smallest work (JSW) and their power of two variants are used in load balancing systems where the instantaneous queue length or workload information at all queues or a subset of them can be queried. In situations where the dispatcher has an associated memory, one can minimize this query overhead by maintaining a list of idle servers to which jobs can be dispatched. Recent alternative approaches that do not require querying such information include the cancel on start and cancel on complete based replication policies. The downside of such policies however is that the servers must communicate the start or completion of each service to the dispatcher and must allow cancellation of redundant copies. In practice, the requirements of query messaging, memory, and replica cancellation pose challenges in their implementation and their advantages are not clear. In this work, we consider load balancing policies that do not query load information, do not have a memory, and do not cancel replicas. Surprisingly, we were able to identify operating regimes where such policies have better performance when compared to some of the popular policies that utilize server feedback information. Our policies allow the dispatcher to append a timer to each job or its replica. A job or a replica is discarded if its timer expires before it starts receiving service. We analyze several variants of this policy which are novel, simple to implement, and also have remarkably good performance in some operating regimes, despite no feedback from servers to the dispatcher.