ICGMM: CXL-enabled Memory Expansion with Intelligent Caching Using Gaussian Mixture Model

TL;DR

This paper introduces ICGMM, a hardware-optimized caching approach using Gaussian Mixture Models to improve memory expansion via CXL, significantly reducing cache miss rates and SSD latency.

Contribution

ICGMM is a novel GMM-based caching and eviction method implemented on FPGA, outperforming traditional strategies in reducing latency and cache misses with fewer hardware resources.

Findings

Cache miss rate reduced by up to 6.14%

SSD access latency decreased by up to 39.14%

GMM-based cache outperforms LRU and LSTM strategies

Abstract

Compute Express Link (CXL) emerges as a solution for wide gap between computational speed and data communication rates among host and multiple devices. It fosters a unified and coherent memory space between host and CXL storage devices such as such as Solid-state drive (SSD) for memory expansion, with a corresponding DRAM implemented as the device cache. However, this introduces challenges such as substantial cache miss penalties, sub-optimal caching due to data access granularity mismatch between the DRAM "cache" and SSD "memory", and inefficient hardware cache management. To address these issues, we propose a novel solution, named ICGMM, which optimizes caching and eviction directly on hardware, employing a Gaussian Mixture Model (GMM)-based approach. We prototype our solution on an FPGA board, which demonstrates a noteworthy improvement compared to the classic Least Recently Used (LRU) cache strategy. We observe a decrease in the cache miss rate ranging from 0.32% to 6.14%, leading to a substantial 16.23% to 39.14% reduction in the average SSD access latency. Furthermore, when compared to the state-of-the-art Long Short-Term Memory (LSTM)-based cache policies, our GMM algorithm on FPGA showcases an impressive latency reduction of over 10,000 times. Remarkably, this is achieved while demanding much fewer hardware resources.