Analysis of Interference between RDMA and Local Access on Hybrid Memory System

TL;DR

This paper investigates the interference between RDMA remote access and local memory access in hybrid memory systems combining DRAM and DCPM, revealing unique challenges and the need for throttling mechanisms.

Contribution

It provides an analysis of interference effects in hybrid memory systems with DCPM and RDMA, highlighting differences from traditional DRAM-only systems and suggesting mitigation strategies.

Findings

Interference in hybrid memory systems differs significantly from DRAM-only systems.

RDMA access can cause performance degradation due to interference.

Throttling mechanisms are necessary to mitigate interference effects.

Abstract

We can use a hybrid memory system consisting of DRAM and Intel Optane DC Persistent Memory (We call it DCPM in this paper) as DCPM is now commercially available since April 2019. Even if the latency for DCPM is several times higher than that for DRAM, the capacity for DCPM is several times higher than that for DRAM and the cost of DCPM is also several times lower than that for DRAM. In addition, DCPM is non-volatile. A Server with this hybrid memory system could improve the performance for in-memory database systems and virtual machine (VM) systems because these systems often consume a large amount of memory. Moreover, a high-speed shared storage system can be implemented by accessing DCPM via remote direct memory access (RDMA). I assume that some of the DCPM is often assigned as a shared area among other remote servers because applications executed on a server with a hybrid memory system often cannot use the entire capacity of DCPM. This paper evaluates the interference between local memory access and RDMA from a remote server. As a result, I indicate that the interference on this hybrid memory system is significantly different from that on a conventional DRAM-only memory system. I also believe that some kind of throttling implementation is needed when this interference occures.