Impact of Distributed Rate Limiting on Load Distribution in a Latency-sensitive Messaging Service

TL;DR

This paper explores how distributed rate limiting affects load distribution in latency-sensitive IoT messaging services, highlighting the trade-offs and proposing a real-time system solution to balance performance and resource control.

Contribution

It identifies the trade-off between load distribution and rate limiting in cloud services and presents a real-time messaging system that effectively manages this balance.

Findings

Demonstrates the impact of rate limiting on load distribution.

Proposes a solution that balances latency and resource constraints.

Validates the approach with a real-world IoT messaging system.

Abstract

The cloud's flexibility and promise of seamless auto-scaling notwithstanding, its ability to meet service level objectives (SLOs) typically calls for some form of control in resource usage. This seemingly traditional problem gives rise to new challenges in a cloud setting, and in particular a subtle yet significant trade-off involving load-distribution decisions (the distribution of workload across available cloud resources to optimize performance), and rate limiting (the capping of individual workloads to prevent global over-commitment). This paper investigates that trade-off through the design and implementation of a real-time messaging system motivated by Internet-of-Things (IoT) applications, and demonstrates a solution capable of realizing an effective compromise. The paper's contributions are in both explicating the source of this trade-off, and in demonstrating a possible solution.