Recent Advancements In Distributed System Communications

TL;DR

This survey reviews recent research (2010-2020) on reducing communication overheads in distributed systems, focusing on hardware and software solutions like RDMA, NICs, kernel bypass, and user space networking.

Contribution

It categorizes and analyzes recent advancements in distributed system communication, highlighting hardware and software approaches to improve performance.

Findings

Hardware-based approaches like RDMA and NICs offer high performance.

Software-based solutions include optimized sockets and user space networking.

Kernel bypass techniques significantly reduce communication overheads.

Abstract

Overheads in Operating System kernel network stacks and sockets have been hindering OSes from managing networking operations efficiently for years. Moreover, when building Remote Procedure Calls over TCP, certain TCP features do not match the needs of RPCs, imposing additional overheads. These issues degrade the performance of distributed systems, which rely on fast communications between machines to be able to serve a large number of client requests with low latency and high throughput. The purpose of this literature survey is to look into recent proposals in research literature that aim to overcome these issues. The survey investigates research literature published between 2010-2020, in order to include important advancements during the most recent decade at the time of writing. The proposals found in papers have been categorized into hardware-based and software-based approaches. The former require specialized hardware to offer high communications performance. The latter are implemented in software and don't rely on specialized hardware or require only certain hardware features. Furthermore, the proposals where also classified according to whether they implement kernel bypass, to avoid using the Operating System kernel network stack, or not. The hardware-based approaches examined here are RDMA, programmable Network Interface Controllers (NIC) and System-on-a-Chip (SoC), while the software-based approaches include optimized socket implementations and RPC frameworks, as well as user space networking.