TailBench++: Flexible Multi-Client, Multi-Server Benchmarking for Latency-Critical Workloads

TL;DR

TailBench++ is an enhanced benchmarking suite designed to evaluate latency-critical cloud workloads in dynamic multi-client, multi-server environments, addressing limitations of existing tools for more realistic performance assessment.

Contribution

This work extends TailBench to support multi-server, variable load, and client arrival patterns, enabling more comprehensive cloud performance evaluations.

Findings

TailBench++ captures a wider range of real-world scenarios.

It enables reproducible experiments with dynamic client and server configurations.

Case studies demonstrate improved evaluation realism.

Abstract

Cloud systems have rapidly expanded worldwide in the last decade, shifting computational tasks to cloud servers where clients submit their requests. Among cloud workloads, latency-critical applications -- characterized by high-percentile response times -- have gained special interest. These applications are present in modern services, representing an important fraction of cloud workloads. This work analyzes common cloud benchmarking suites and identifies TailBench as the most suitable to assess cloud performance with latency-critical workloads. Unfortunately, this suite presents key limitations, especially in multi-server scenarios or environments with variable client arrival patterns and fluctuating loads. To address these limitations, we propose TailBench++, an enhanced benchmark suite that extends TailBench to enable cloud evaluation studies to be performed in dynamic multi-client, multi-server environments. It allows reproducing experiments with varying client arrival times, dynamic query per second (QPS) fluctuations, and multiple servers handling requests. Case studies show that TailBench++ enables more realistic evaluations by capturing a wider range of real-world scenarios.