A Survey of Distributed Message Broker Queues

TL;DR

This paper reviews popular distributed message broker queues, comparing Kafka and AMQP protocols in terms of features and performance to guide their practical application in fault-tolerant distributed systems.

Contribution

It provides an empirical comparison of Kafka and AMQP, highlighting their differences in features and performance under various workloads.

Findings

Kafka and AMQP differ significantly in feature sets.

Performance varies based on workload type and system configuration.

Insights assist in selecting suitable message brokers for distributed architectures.

Abstract

This paper surveys the message brokers that are in vogue today for distributed communication. Their primary goal is to facilitate the construction of decentralized topologies without single points of failure, enabling fault tolerance and high availability. These characteristics make them optimal for usage within distributed architectures. However, there are multiple protocols built to achieve this, and it would be beneficial to have a empirical comparison between their features and performance to determine their real-world applicability. This paper focuses on two popular protocols (Kafka and AMQP) and explores the divergence in their features as well as their performance under varied testing workloads.