EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

TL;DR

EDM introduces an Ethernet-based memory disaggregation fabric that significantly reduces remote memory access latency by implementing the entire protocol stack within the PHY and using a centralized scheduler, achieving near PCIe-level latency.

Contribution

This paper presents EDM, a novel Ethernet fabric that implements remote memory access protocols within the PHY and employs a fast in-network scheduler to drastically lower latency.

Findings

Latency of ~300 ns for remote memory access in FPGA testbed

Maintains low latency (~1.3x unloaded) even at high network loads

Comparable latency to PCIe-based solutions like CXL

Abstract

Achieving low remote memory access latency remains the primary challenge in realizing memory disaggregation over Ethernet within the datacenters. We present EDM that attempts to overcome this challenge using two key ideas. First, while existing network protocols for remote memory access over the Ethernet, such as TCP/IP and RDMA, are implemented on top of the MAC layer, EDM takes a radical approach by implementing the entire network protocol stack for remote memory access within the Physical layer (PHY) of the Ethernet. This overcomes fundamental latency and bandwidth overheads imposed by the MAC layer, especially for small memory messages. Second, EDM implements a centralized, fast, in-network scheduler for memory traffic within the PHY of the Ethernet switch. Inspired by the classic Parallel Iterative Matching (PIM) algorithm, the scheduler dynamically reserves bandwidth between compute and memory nodes by creating virtual circuits in the PHY, thus eliminating queuing delay and layer 2 packet processing delay at the switch for memory traffic, while maintaining high bandwidth utilization. Our FPGA testbed demonstrates that EDM's network fabric incurs a latency of only $\sim$300 ns for remote memory access in an unloaded network, which is an order of magnitude lower than state-of-the-art Ethernet-based solutions such as RoCEv2 and comparable to emerging PCIe-based solutions such as CXL. Larger-scale network simulations indicate that even at high network loads, EDM's average latency remains within 1.3$\times$ its unloaded latency.