Space-Optimal, Computation-Optimal, Topology-Agnostic, Throughput-Scalable Causal Delivery through Hybrid Buffering

TL;DR

This paper introduces a novel topology-agnostic causal message delivery algorithm that combines sender and receiver buffering, achieving optimal throughput, minimal metadata overhead, and computational efficiency in distributed systems.

Contribution

The paper presents a new hybrid causal delivery algorithm that overcomes limitations of previous sender-only buffering methods, ensuring scalability and efficiency.

Findings

Achieves effectively constant metadata size per message.

Ensures computationally-optimal processing overhead.

Provides topology-agnostic causal delivery with high scalability.

Abstract

Message delivery respecting causal ordering (causal delivery) is one of the most classic and widely useful abstraction for inter-process communication in a distributed system. Most approaches tag messages with causality information and buffer them at the receiver until they can be safely delivered. Except for specific approaches that exploit communication topology, therefore not generally applicable, they incur a metadata overhead which is prohibitive for a large number of processes. Much less used are the approaches that enforce causal order by buffering messages at the sender, until it is safe to release them to the network, as the classic algorithm has too many drawbacks. In this paper, first we discuss the limitations of sender-only buffering approaches and introduce the Sender Permission to Send (SPS) enforcement strategy, showing that SPS + FIFO implies Causal. We analyze a recent sender-buffering algorithm, Cykas, which follows SPS + FIFO, albeit very conservatively, pointing out throughput scalability and liveness issues. Then, we introduce a novel SPS + FIFO based algorithm, which adopts a new hybrid approach: enforcing causality by combining sender-buffering to enforce SPS and receiver-buffering to enforce FIFO. The algorithm overcomes limitations of sender-only buffering, and achieves effectively constant metadata size per message. By a careful choice of data-structures, the algorithm is also computationally-optimal, with amortized effectively constant processing overhead. As far as we know, there is no other topology-agnostic causal delivery algorithm with these properties.