Using Intel Optane Devices for In-situ Data Staging in HPC Workflows

TL;DR

This paper evaluates Intel Optane's performance as an in-situ data staging device in HPC workflows, comparing it to NVMe drives through benchmarking and integration into scientific data management frameworks.

Contribution

It provides a comprehensive performance comparison of Optane and NVMe drives and demonstrates Optane's effectiveness in in-memory data staging for scientific workflows.

Findings

Optane shows lower latency than NVMe in benchmarks.

Using Optane improves data staging efficiency in scientific workflows.

Performance gains depend on workload characteristics.

Abstract

Emerging non-volatile memory technologies (NVRAM) offer alternatives to hard drives that are persistent, while providing similar latencies to DRAM. Intel recently released the Optane drive, which features 3D XPoint memory technology. This device can be deployed as an SSD or as persistent memory. In this paper, we provide a performance comparison between Optane (SSD DC4800X) and NVMe (SSD DC3700) drives as block devices. We study the performance from two perspectives: 1) Benchmarking of drives using FIO workloads, and 2) Assessing the impact of using Optane over NVMe within the DataSpaces framework for in-memory data staging to support in-situ scientific workflows.