xNVMe: Unleashing Storage Hardware-Software Co-design

TL;DR

xNVMe is a unified, open-source API that simplifies and standardizes access to NVMe storage hardware across different operating systems and libraries, facilitating easier development and broader adoption.

Contribution

The paper introduces xNVMe, a single message-passing API that unifies diverse storage I/O paths for NVMe devices across multiple OS and libraries, with minimal overhead.

Findings

xNVMe supports multiple OS and libraries with minimal overhead

It has gained industry traction as an open-source project

The paper shares lessons learned and future plans for xNVMe

Abstract

NVMe SSD hardware has witnessed widespread deployment as commodity and enterprise hardware due to its high performance and rich feature set. Despite the open specifications of various NVMe protocols by the NVMe Express group and NVMe being of software abstractions to program the underlying hardware. The myriad storage I/O paths such as POSIX storage API, ad-hoc OS mechanisms, and userspace I/O libraries have different syntax and semantics that complicate software development and stand in the way of mass adoption and evolution of the NVMe ecosystem. To unify the diverse I/O storage paths, we built xNVMe that exposes a single message-passing API to support both asynchronous and synchronous communication with NVMe devices. xNVMe provides various command sets to support diverse storage I/O paths in different OS (e.g., Linux, FreeBSD, Windows, and MacOS) and userspace libraries (e.g., SPDK) with minimal overhead. xNVMe is an Open Source project and has gained traction amongst various industry stakeholders. In this paper, we elaborate on the lessons that we have learned in the project during its evolution. We also provide some ongoing and future work planned for the project. We hope the database and storage systems community can join in the effort to both extend xNVMe and leverage it as a building block for innovative co-design of storage systems on modern NVMe hardware.