RDMA is Turing complete, we just did not know it yet!

TL;DR

This paper demonstrates that RDMA can be made Turing complete through self-modifying chains, enabling complex offloads and significantly improving performance and resilience in distributed systems without hardware changes.

Contribution

We introduce RedN, a novel approach that achieves Turing completeness in RDMA offloads using self-modifying chains, expanding RDMA capabilities without hardware modifications.

Findings

RedN reduces key-value get latency by up to 2.6x.

RedN offers performance isolation and reduces latency by up to 35x under contention.

RedN enables complex offloads like key lookups in applications such as Memcached.

Abstract

It is becoming increasingly popular for distributed systems to exploit offload to reduce load on the CPU. Remote Direct Memory Access (RDMA) offload, in particular, has become popular. However, RDMA still requires CPU intervention for complex offloads that go beyond simple remote memory access. As such, the offload potential is limited and RDMA-based systems usually have to work around such limitations. We present RedN, a principled, practical approach to implementing complex RDMA offloads, without requiring any hardware modifications. Using self-modifying RDMA chains, we lift the existing RDMA verbs interface to a Turing complete set of programming abstractions. We explore what is possible in terms of offload complexity and performance with a commodity RDMA NIC. We show how to integrate these RDMA chains into applications, such as the Memcached key-value store, allowing us to offload complex tasks such as key lookups. RedN can reduce the latency of key-value get operations by up to 2.6x compared to state-of-the-art KV designs that use one-sided RDMA primitives (e.g., FaRM-KV), as well as traditional RPC-over-RDMA approaches. Moreover, compared to these baselines, RedN provides performance isolation and, in the presence of contention, can reduce latency by up to 35x while providing applications with failure resiliency to OS and process crashes.