Offloading to CXL-based Computational Memory

TL;DR

This paper introduces KAI, a system leveraging CXL-based computational memory with an asynchronous back-streaming protocol, significantly reducing runtime and idle times for data-intensive workloads.

Contribution

It proposes a novel asynchronous back-streaming protocol and a system implementation, KAI, to optimize data movement and processing in CXL-based computational memory systems.

Findings

KAI reduces end-to-end runtime by up to 50.4%.

CCM and host idle times are decreased by 22.11x and 3.85x on average.

The approach demonstrates improved system efficiency for diverse workloads.

Abstract

CXL-based Computational Memory (CCM) enables near-memory processing within expanded remote memory, presenting opportunities to address data movement costs associated with disaggregated memory systems and to accelerate overall performance. However, existing operation offloading mechanisms are not capable of leveraging the trade-offs of different models based on different CXL protocols. This work first examines these tradeoffs and demonstrates their impact on end-to-end performance and system efficiency for workloads with diverse data and processing requirements. We propose a novel 'Asynchronous Back-Streaming' protocol by carefully layering data and control transfer operations on top of the underlying CXL protocols. We design KAI, a system that realizes the asynchronous back-streaming model that supports asynchronous data movement and lightweight pipelining in host-CCM interactions. Overall, KAI reduces end-to-end runtime by up to 50.4%, and CCM and host idle times by average 22.11x and 3.85x, respectively.