A Configurable Transport Layer for CAF

TL;DR

This paper proposes a flexible, configurable transport layer for actor systems, decoupling messaging guarantees from transport protocols to improve portability, transparency, and adaptability in distributed actor frameworks.

Contribution

It introduces a modular design for actor communication, enabling arbitrary transport protocols to be combined with messaging guarantees in the CAF framework.

Findings

Decoupling messaging guarantees from transport protocols is feasible.

The redesigned network stack maintains performance while increasing flexibility.

Layer composability impacts overall system costs and performance.

Abstract

The message-driven nature of actors lays a foundation for developing scalable and distributed software. While the actor itself has been thoroughly modeled, the message passing layer lacks a common definition. Properties and guarantees of message exchange often shift with implementations and contexts. This adds complexity to the development process, limits portability, and removes transparency from distributed actor systems. In this work, we examine actor communication, focusing on the implementation and runtime costs of reliable and ordered delivery. Both guarantees are often based on TCP for remote messaging, which mixes network transport with the semantics of messaging. However, the choice of transport may follow different constraints and is often governed by deployment. As a first step towards re-architecting actor-to-actor communication, we decouple the messaging guarantees from the transport protocol. We validate our approach by redesigning the network stack of the C++ Actor Framework (CAF) so that it allows to combine an arbitrary transport protocol with additional functions for remote messaging. An evaluation quantifies the cost of composability and the impact of individual layers on the entire stack.