Jiffy: A Fast, Memory Efficient, Wait-Free Multi-Producers Single-Consumer Queue

TL;DR

Jiffy is a novel wait-free multi-producer single-consumer queue that offers high throughput and significantly reduced memory usage, outperforming existing solutions in scalability and efficiency.

Contribution

The paper introduces Jiffy, a new memory-efficient wait-free multi-producer single-consumer queue with formal correctness proof and superior performance.

Findings

Jiffy achieves up to 50% higher throughput than comparable queues.

Jiffy uses approximately 90% less memory than other state-of-the-art queues.

Jiffy's performance remains robust with up to 128 concurrent threads.

Abstract

In applications such as sharded data processing systems, sharded in-memory key-value stores, data flow programming and load sharing applications, multiple concurrent data producers are feeding requests into the same data consumer. This can be naturally realized through concurrent queues, where each consumer pulls its tasks from its dedicated queue. For scalability, wait-free queues are often preferred over lock based structures. The vast majority of wait-free queue implementations, and even lock-free ones, support the multi-producer multi-consumer model. Yet, this comes at a premium, since implementing wait-free multi-producer multi-consumer queues requires utilizing complex helper data structures. The latter increases the memory consumption of such queues and limits their performance and scalability. Additionally, many such designs employ (hardware) cache unfriendly memory access patterns. In this work we study the implementation of wait-free multi-producer single-consumer queues. Specifically, we propose Jiffy, an efficient memory frugal novel wait-free multi-producer single-consumer queue and formally prove its correctness. We then compare the performance and memory requirements of Jiffy with other state of the art lock-free and wait-free queues. We show that indeed Jiffy can maintain good performance with up to 128 threads, delivers up to 50% better throughput than the next best construction we compared against, and consumes ~90% less memory.