A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications

TL;DR

Atlas introduces a hybrid data plane for far memory that dynamically switches between paging and object fetching, optimizing efficiency and latency based on workload locality.

Contribution

It presents a runtime-kernel co-designed system that adaptively combines paging and object fetching for far memory, improving performance and resource utilization.

Findings

Improves throughput by up to 3.2x over state-of-the-art methods.

Reduces tail latency by one to two orders of magnitude.

Dynamically adapts data fetching strategy based on workload locality.

Abstract

With rapid advances in network hardware, far memory has gained a great deal of traction due to its ability to break the memory capacity wall. Existing far memory systems fall into one of two data paths: one that uses the kernel's paging system to transparently access far memory at the page granularity, and a second that bypasses the kernel, fetching data at the object granularity. While it is generally believed that object fetching outperforms paging due to its fine-grained access, it requires significantly more compute resources to run object-level LRU and eviction. We built Atlas, a hybrid data plane enabled by a runtime-kernel co-design that simultaneously enables accesses via these two data paths to provide high efficiency for real-world applications. Atlas uses always-on profiling to continuously measure page locality. For workloads already with good locality, paging is used to fetch data, whereas for those without, object fetching is employed. Object fetching moves objects that are accessed close in time to contiguous local space, dynamically improving locality and making the execution increasingly amenable to paging, which is much more resource-efficient. Our evaluation shows that Atlas improves the throughput (e.g., by 1.5x and 3.2x) and reduces the tail latency (e.g., by one and two orders of magnitude) when using remote memory, compared with AIFM and Fastswap, the state-of-the-art techniques respectively in the two categories.