DTranx: A SEDA-based Distributed and Transactional Key Value Store with Persistent Memory Log

TL;DR

DTranx is a distributed key-value store leveraging persistent memory and advanced concurrency techniques to achieve high throughput and efficiency without sacrificing consistency.

Contribution

It introduces a persistent memory aware log, a state transition based garbage collection, and a hybrid commit protocol for improved performance and consistency.

Findings

Achieves 514.11k transactions/sec with 36 servers and 95% read workload.

Persistent memory log is 30 times faster than SSD-based systems.

Garbage collection is efficient, non-disruptive, and maintains low log space usage.

Abstract

Current distributed key value stores achieve scalability by trading off consistency. As persistent memory technologies evolve tremendously, it is not necessary to sacrifice consistency for performance. This paper proposes DTranx, a distributed key value store based on a persistent memory aware log. DTranx integrates a state transition based garbage collection mechanism in the log design to effectively and efficiently reclaim old logs. In addition, DTranx adopts the SEDA architecture to exploit higher concurrency in multi-core environments and employs the optimal core binding strategy to minimize context switch overhead. Moreover, we customize a hybrid commit protocol that combines optimistic concurrency control and two-phase commit to reduce critical section of distributed locking and introduce a locking mechanism to avoid deadlocks and livelocks. In our evaluations, DTranx reaches 514.11k transactions per second with 36 servers and 95\% read workloads. The persistent memory aware log is 30 times faster than the SSD based system. And, our state transition based garbage collection mechanism is efficient and effective. It does not affect normal transactions and log space usage is steadily low.