HybridTier: an Adaptive and Lightweight CXL-Memory Tiering System

TL;DR

HybridTier is a novel adaptive CXL-memory tiering system that efficiently tracks data hotness using probabilistic methods, significantly improving performance and reducing overhead compared to prior approaches.

Contribution

It introduces HybridTier, which adaptively tracks data hotness using probabilistic metadata and lightweight data structures, addressing dynamic hotness shifts with minimal overhead.

Findings

HybridTier outperforms prior systems by up to 91% in performance.

It reduces memory overhead by 2.0-7.8 times.

It decreases cache misses by 1.7-3.5 times.

Abstract

Modern workloads are demanding increasingly larger memory capacity. Compute Express Link (CXL)-based memory tiering has emerged as a promising solution for addressing this problem by utilizing traditional DRAM alongside slow-tier CXL memory devices. We analyze prior tiering systems and observe two challenges for high-performance memory tiering: adapting to skewed but dynamically varying data hotness distributions while minimizing memory and cache overhead due to tiering. To address these challenges, we propose HybridTier, an adaptive and lightweight tiering system for CXL memory. HybridTier tracks both long-term data access frequency and short-term access momentum \emph{simultaneously} to accurately capture and adapt to shifting hotness distributions. HybridTier reduces the metadata memory overhead by tracking data accesses \emph{probabilistically}, obtaining higher memory efficiency by trading off a small amount of tracking inaccuracy that has a negligible impact on application performance. To reduce cache overhead, HybridTier uses lightweight data structures that optimize for data locality to track data hotness. Our evaluations show that HybridTier outperforms prior systems by up to $91\%$ ($19\%$ geomean), incurring $2.0-7.8\times$ less memory overhead and $1.7-3.5\times$ less cache misses.