WiscSort: External Sorting For Byte-Addressable Storage

TL;DR

WiscSort is a high-performance external sorting algorithm optimized for byte-addressable storage devices, reducing I/O and improving speed through novel scheduling and key-value separation techniques.

Contribution

WiscSort introduces the BRAID model and new concurrency strategies to enhance sorting performance on byte-addressable storage devices.

Findings

WiscSort is 2-7x faster than existing methods on standard benchmarks.

Key-value separation improves sorting efficiency across different sizes.

The approach performs well across various hardware configurations.

Abstract

We present WiscSort, a new approach to high-performance concurrent sorting for existing and future byte-addressable storage (BAS) devices. WiscSort carefully reduces writes, exploits random reads by splitting keys and values during sorting, and performs interference-aware scheduling with thread pool sizing to avoid I/O bandwidth degradation. We introduce the BRAID model which encompasses the unique characteristics of BAS devices. Many state-of-the-art sorting systems do not comply with the BRAID model and deliver sub-optimal performance, whereas WiscSort demonstrates the effectiveness of complying with BRAID. We show that WiscSort is 2-7x faster than competing approaches on a standard sort benchmark. We evaluate the effectiveness of key-value separation on different key-value sizes and compare our concurrency optimizations with various other concurrency models. Finally, we emulate generic BAS devices and show how our techniques perform well with various combinations of hardware properties.