Faster than Flash: An In-Depth Study of System Challenges for Emerging Ultra-Low Latency SSDs

TL;DR

This paper provides a comprehensive empirical analysis of ultra-low latency SSDs, evaluating I/O methods and system architectures to identify challenges and guide future system optimizations for emerging storage technologies.

Contribution

It offers the first quantitative study of ULL SSD behaviors, analyzing I/O completion methods, system resource usage, and real-world challenges in SPDK-enabled environments.

Findings

Polling I/O methods show different efficiencies depending on system parameters.

UL SSDs face significant challenges in resource management and system integration.

SPDK-based systems require careful tuning to fully leverage ULL SSD capabilities.

Abstract

Emerging storage systems with new flash exhibit ultra-low latency (ULL) that can address performance disparities between DRAM and conventional solid state drives (SSDs) in the memory hierarchy. Considering the advanced low-latency characteristics, different types of I/O completion methods (polling/hybrid) and storage stack architecture (SPDK) are proposed. While these new techniques are expected to take costly software interventions off the critical path in ULL-applied systems, unfortunately no study exists to quantitatively analyze system-level characteristics and challenges of combining such newly-introduced techniques with real ULL SSDs. In this work, we comprehensively perform empirical evaluations with 800GB ULL SSD prototypes and characterize ULL behaviors by considering a wide range of I/O path parameters, such as different queues and access patterns. We then analyze the efficiencies and challenges of the polled-mode and hybrid polling I/O completion methods (added into Linux kernels 4.4 and 4.10, respectively) and compare them with the efficiencies of a conventional interrupt-based I/O path. In addition, we revisit the common expectations of SPDK by examining all the system resources and parameters. Finally, we demonstrate the challenges of ULL SSDs in a real SPDK-enabled server-client system. Based on the performance behaviors that this study uncovers, we also discuss several system implications, which are required to take a full advantage of ULL SSD in the future.