Skueue: A Scalable and Sequentially Consistent Distributed Queue

TL;DR

Skueue is a distributed queue protocol that ensures sequential consistency, scales efficiently by batching requests, and adapts to dynamic environments with high throughput, avoiding bottlenecks in asynchronous message passing systems.

Contribution

It introduces Skueue, a scalable, sequentially consistent distributed queue protocol that handles dynamic environments and high request rates efficiently.

Findings

Operations require logarithmic rounds with high probability.

Skueue maintains queue semantics under high load.

The protocol adapts to dynamic process membership.

Abstract

We propose a distributed protocol for a queue, called \textsc{Skueue}, which spreads its data fairly onto multiple processes, avoiding bottlenecks in high throughput scenarios. \textsc{Skueue} can be used in highly dynamic environments, through the addition of join and leave requests to the standard queue operations enqueue and dequeue. Furthermore \textsc{Skueue} satisfies sequential consistency in the asynchronous message passing model. Scalability is achieved by aggregating multiple requests to a batch, which can then be processed in a distributed fashion without hurting the queue semantics. Operations in \textsc{Skueue} need a logarithmic number of rounds w.h.p. until they are processed, even under a high rate of incoming requests.