Evaluating Emerging CXL-enabled Memory Pooling for HPC Systems

TL;DR

This paper evaluates CXL-enabled memory pooling for HPC systems, demonstrating potential performance benefits for diverse workloads and identifying practical challenges like interference in shared pools.

Contribution

It introduces an emulator and profiler to assess CXL memory pooling, exploring capacity and bandwidth provisioning for HPC workloads, and highlights practical adoption challenges.

Findings

Less than 10% performance impact for some scientific applications with 75% pooled memory

High-bandwidth configurations support bandwidth-intensive applications effectively

Shared memory interference poses practical challenges for deployment

Abstract

Current HPC systems provide memory resources that are statically configured and tightly coupled with compute nodes. However, workloads on HPC systems are evolving. Diverse workloads lead to a need for configurable memory resources to achieve high performance and utilization. In this study, we evaluate a memory subsystem design leveraging CXL-enabled memory pooling. Two promising use cases of composable memory subsystems are studied -- fine-grained capacity provisioning and scalable bandwidth provisioning. We developed an emulator to explore the performance impact of various memory compositions. We also provide a profiler to identify the memory usage patterns in applications and their optimization opportunities. Seven scientific and six graph applications are evaluated on various emulated memory configurations. Three out of seven scientific applications had less than 10% performance impact when the pooled memory backed 75% of their memory footprint. The results also show that a dynamically configured high-bandwidth system can effectively support bandwidth-intensive unstructured mesh-based applications like OpenFOAM. Finally, we identify interference through shared memory pools as a practical challenge for adoption on HPC systems.