Heterogeneous Memory Pool Tuning

TL;DR

This paper introduces a lightweight, non-intrusive tool for analyzing and tuning data placement in systems with heterogeneous memory pools, demonstrating how optimal placement impacts performance on Intel Sapphire Rapids with HBM and DDR.

Contribution

The paper presents a novel tool for analyzing and tuning application data placement in heterogeneous memory systems, enabling performance optimization without intrusive modifications.

Findings

Only 60-75% of data needs to be in HBM to reach 90% of maximum performance.

The tool effectively analyzes data placement and system performance metrics.

Performance varies significantly based on data placement strategies.

Abstract

We present a lightweight tool for the analysis and tuning of application data placement in systems with heterogeneous memory pools. The tool allows non-intrusively identifying, analyzing, and controlling the placement of individual allocations of the application. We use the tool to analyze a set of benchmarks running on the Intel Sapphire Rapids platform with both HBM and DDR memory. The paper also contains an analysis of the performance of both memory subsystems in terms of read/write bandwidth and latency. The key part of the analysis is to focus on performance if both subsystems are used together. We show that only about 60% to 75% of the data must be placed in HBM memory to achieve 90% of the potential performance of the platform on those benchmarks.