Performance Evaluation of Brokerless Messaging Libraries

TL;DR

This paper systematically evaluates the performance of brokerless messaging libraries like ZeroMQ, NanoMsg, and NNG through extensive benchmarking, providing insights to help practitioners choose the most suitable solution.

Contribution

It introduces a comprehensive benchmarking suite for brokerless messaging systems and offers a comparative analysis of leading libraries, filling a gap in performance evaluation research.

Findings

ZeroMQ outperforms NanoMsg and NNG in high-throughput scenarios.

Different libraries have distinct strengths depending on workload conditions.

The benchmarking suite enables fair and systematic evaluation of brokerless messaging systems.

Abstract

Messaging systems are essential for efficiently transferring large volumes of data, ensuring rapid response times and high-throughput communication. The state-of-the-art on messaging systems mainly focuses on the performance evaluation of brokered messaging systems, which use an intermediate broker to guarantee reliability and quality of service. However, over the past decade, brokerless messaging systems have emerged, eliminating the single point of failure and trading off reliability guarantees for higher performance. Still, the state-of-the-art on evaluating the performance of brokerless systems is scarce. In this work, we solely focus on brokerless messaging systems. First, we perform a qualitative analysis of several possible candidates, to find the most promising ones. We then design and implement an extensive open-source benchmarking suite to systematically and fairly evaluate the performance of the chosen libraries, namely, ZeroMQ, NanoMsg, and NanoMsg-Next-Generation (NNG). We evaluate these libraries considering different metrics and workload conditions, and provide useful insights into their limitations. Our analysis enables practitioners to select the most suitable library for their requirements.