CXLAimPod: CXL Memory is all you need in AI era

TL;DR

CXLAimPod leverages CXL's full-duplex memory channels with adaptive, application-aware scheduling to significantly improve performance for data-intensive AI and database workloads, surpassing traditional DDR5 architectures.

Contribution

The paper introduces CXLAimPod, a novel adaptive scheduling framework that exploits CXL's full-duplex capabilities through system support and application hints, enhancing memory system performance.

Findings

CXL systems achieve 55-61% bandwidth improvement over DDR5 at balanced read-write ratios.

CXLAimPod improves Redis performance by 7.4% on average, up to 150% in specific cases.

It delivers 71.6% improvement for LLM text generation and 9.1% for vector databases.

Abstract

The proliferation of data-intensive applications, ranging from large language models to key-value stores, increasingly stresses memory systems with mixed read-write access patterns. Traditional half-duplex architectures such as DDR5 are ill-suited for such workloads, suffering bus turnaround penalties that reduce their effective bandwidth under mixed read-write patterns. Compute Express Link (CXL) offers a breakthrough with its full-duplex channels, yet this architectural potential remains untapped as existing software stacks are oblivious to this capability. This paper introduces CXLAimPod, an adaptive scheduling framework designed to bridge this software-hardware gap through system support, including cgroup-based hints for application-aware optimization. Our characterization quantifies the opportunity, revealing that CXL systems achieve 55-61% bandwidth improvement at balanced read-write ratios compared to flat DDR5 performance, demonstrating the benefits of full-duplex architecture. To realize this potential, the CXLAimPod framework integrates multiple scheduling strategies with a cgroup-based hint mechanism to navigate the trade-offs between throughput, latency, and overhead. Implemented efficiently within the Linux kernel via eBPF, CXLAimPod delivers significant performance improvements over default CXL configurations. Evaluation on diverse workloads shows 7.4% average improvement for Redis (with up to 150% for specific sequential patterns), 71.6% improvement for LLM text generation, and 9.1% for vector databases, demon-strating that duplex-aware scheduling can effectively exploit CXL's architectural advantages.