Pushing the Memory Bandwidth Wall with CXL-enabled Idle I/O Bandwidth Harvesting

TL;DR

This paper introduces SURGE, a technique that uses CXL technology to dynamically multiplex idle I/O bandwidth with memory traffic, significantly improving memory bandwidth utilization for bandwidth-constrained servers.

Contribution

The paper proposes SURGE, a novel software-supported architecture that enhances memory bandwidth by multiplexing I/O and memory traffic over CXL, addressing bandwidth fragmentation issues.

Findings

Up to 1.3x acceleration of memory-intensive workloads

Effective utilization of idle I/O bandwidth resources

Demonstrates benefits of CXL-enabled multiplexing

Abstract

The continual increase of cores on server-grade CPUs raises demands on memory systems, which are constrained by limited off-chip pin and data transfer rate scalability. As a result, high-end processors typically feature lower memory bandwidth per core, at the detriment of memory-intensive workloads. We propose alleviating this challenge by improving the utility of the CPU's limited pins. In a typical CPU design process, the available pins are apportioned between memory and I/O traffic, each accounting for about half of the total off-chip bandwidth availability. Consequently, unless both memory and I/O are simultaneously highly utilized, such fragmentation leads to underutilization of the valuable off-chip bandwidth resources. An ideal architecture would offer I/O and memory bandwidth fungibility, allowing use of the aggregate off-chip bandwidth in the form required by each workload. In this work, we introduce SURGE, a software-supported architectural technique that boosts memory bandwidth availability by salvaging idle I/O bandwidth resources. SURGE leverages the capability of versatile interconnect technologies like CXL to dynamically multiplex memory and I/O traffic over the same processor interface. We demonstrate that SURGE-enhanced architectures can accelerate memory-intensive workloads on bandwidth-constrained servers by up to 1.3x.