Bridging Cache-Friendliness and Concurrency: A Locality-Optimized In-Memory B-Skiplist

TL;DR

This paper introduces a cache-friendly, concurrent B-skiplist that significantly improves throughput and latency over traditional skiplists and competes well with tree-based indices, enhancing in-memory key-value store performance.

Contribution

It presents a novel top-down insertion algorithm and concurrency control for B-skiplists that improve cache locality and performance while maintaining simplicity.

Findings

Achieves 2x-9x higher throughput than state-of-the-art skiplists on 128 threads.

Attains 0.9x-1.7x throughput of cache-optimized trees like Masstree.

Reduces 99% latency by 3.5x-103x compared to other skiplists and 0.85x-64x compared to tree indices.

Abstract

Skiplists are widely used for in-memory indexing in many key-value stores, such as RocksDB and LevelDB, due to their ease of implementation and simple concurrency control mechanisms. However, traditional skiplists suffer from poor cache locality, as they store only a single element per node, leaving performance on the table. Minimizing last-level cache misses is key to maximizing in-memory index performance, making high cache locality essential. In this paper, we present a practical concurrent B-skiplist that enhances cache locality and performance while preserving the simplicity of traditional skiplist structures and concurrency control schemes. Our key contributions include a top-down, single-pass insertion algorithm for B-skiplists and a corresponding simple and efficient top-down concurrency control scheme. On 128 threads, the proposed concurrent B-skiplist achieves between 2x-9x higher throughput compared to state-of-the-art concurrent skiplist implementations, including Facebook's concurrent skiplist from Folly and the Java ConcurrentSkipListMap. Furthermore, we find that the B-skiplist achieves competitive (0.9x-1.7x) throughput on point workloads compared to state-of-the-art cache-optimized tree-based indices (e.g., Masstree). For a more complete picture of the performance, we also measure the latency of skiplist and tree-based indices and find that the B-skiplist achieves between 3.5x-103x lower 99% latency compared to other concurrent skiplists and between 0.85x-64x lower 99% latency compared to tree-based indices on point workloads with inserts.