DAOS as HPC Storage, a view from Numerical Weather Prediction

TL;DR

This paper assesses Intel's DAOS object storage system for HPC, demonstrating its scalable performance and introducing a new benchmark to evaluate object storage in data-intensive applications.

Contribution

It provides a preliminary evaluation of DAOS for HPC storage, highlighting its scalability and performance characteristics with a new I/O benchmark.

Findings

DAOS can achieve linear bandwidth scaling with more server nodes

Configuration and application design influence achievable bandwidth

Introduces a new benchmark for object storage performance evaluation

Abstract

Object storage solutions potentially address long-standing performance issues with POSIX file systems for certain I/O workloads, and new storage technologies offer promising performance characteristics for data-intensive use cases. In this work, we present a preliminary assessment of Intel's Distributed Asynchronous Object Store (DAOS), an emerging high-performance object store, in conjunction with non-volatile storage and evaluate its potential use for HPC storage. We demonstrate DAOS can provide the required performance, with bandwidth scaling linearly with additional DAOS server nodes in most cases, although choices in configuration and application design can impact achievable bandwidth. We describe a new I/O benchmark and associated metrics that address object storage performance from application-derived workloads.