Fast Nonblocking Persistence for Concurrent Data Structures

TL;DR

This paper introduces nbMontage, a lock-free, wait-free persistence system for concurrent data structures that guarantees consistency, reduces latency, and achieves performance close to non-persistent counterparts.

Contribution

It extends the Montage system with a fully lock-free, wait-free persistence mechanism that ensures bounded data loss and improved latency without significant overhead.

Findings

Performance rivals custom data structures in literature

Dramatically faster than prior general-purpose systems

Within 50% of non-persistent in-memory structures

Abstract

We present a fully lock-free variant of the recent Montage system for persistent data structures. Our variant, nbMontage, adds persistence to almost any nonblocking concurrent structure without introducing significant overhead or blocking of any kind. Like its predecessor, nbMontage is buffered durably linearizable: it guarantees that the state recovered in the wake of a crash will represent a consistent prefix of pre-crash execution. Unlike its predecessor, nbMontage ensures wait-free progress of the persistence frontier, thereby bounding the number of recent updates that may be lost on a crash, and allowing a thread to force an update of the frontier (i.e., to perform a sync operation) without the risk of blocking. As an extra benefit, the helping mechanism employed by our wait-free sync significantly reduces its latency. Performance results for nonblocking queues, skip lists, trees, and hash tables rival custom data structures in the literature -- dramatically faster than achieved with prior general-purpose systems, and generally within 50% of equivalent non-persistent structures placed in DRAM.