CXLMemUring: A Hardware Software Co-design Paradigm for Asynchronous and Flexible Parallel CXL Memory Pool Access

TL;DR

This paper introduces CXLMemUring, a hardware-software co-design paradigm that enhances asynchronous, flexible parallel CXL memory pool access by offloading memory operations to CXL endpoints and near-device cores, improving memory capacity and efficiency.

Contribution

It proposes a novel hardware-software co-design approach for CXL memory access, including adaptive code generation and profiling-guided updates for long-running jobs.

Findings

Evaluation with modified BOOMv3 shows promising results.

Simulation of CXL endpoint access demonstrates feasibility.

Adaptive code offloading improves memory operation efficiency.

Abstract

CXL has been the emerging technology for expanding memory for both the host CPU and device accelerators with load/store interface. Extending memory coherency to the PCIe root complex makes the codesign more flexible in that you can access the memory with coherency using your near-device computability. Since the capacity demand with tolerable latency and bandwidth is growing, we need to come up with a new hardware-software codesign way to offload the synthesized memory operations to the CXL endpoint, CXL switch or near CXL root complex cores like Intel DSA to fetch data; the CPU or accelerators can calculate other stuff in the backend. On CXL done loading, the data will be put into L1 if capacity fits, and the in-core ROB will be notified by mailbox and resume the calculation on the previous hardware context. Since the distance(timing window) of the load instruction sequence is unknown, a profiling-guided way of codegening and adaptively updating offloaded code will be required for a long-running job. We propose to evaluate CXLMemUring the modified BOOMv3 with added in-core-logic and CXL endpoint access simulation using CHI, and we will add a weaker RISCV Core near endpoint for code offloading, and the codegening will be based on program analysis with traditional profiling guided way.