Run Time Approximation of Non-blocking Service Rates for Streaming Systems

TL;DR

This paper presents an online algorithm to approximate non-blocking service rates of kernels in streaming systems, enabling dynamic tuning without steady state assumptions, validated within the RaftLib framework.

Contribution

It introduces a novel online method for estimating non-blocking service rates in streaming systems, addressing the limitations of static approaches in dynamic environments.

Findings

Algorithm successfully estimates service rates in real-time.

Validated with microbenchmark and real streaming applications.

Enables adaptive optimization of streaming systems.

Abstract

Stream processing is a compute paradigm that promises safe and efficient parallelism. Modern big-data problems are often well suited for stream processing's throughput-oriented nature. Realization of efficient stream processing requires monitoring and optimization of multiple communications links. Most techniques to optimize these links use queueing network models or network flow models, which require some idea of the actual execution rate of each independent compute kernel within the system. What we want to know is how fast can each kernel process data independent of other communicating kernels. This is known as the "service rate" of the kernel within the queueing literature. Current approaches to divining service rates are static. Modern workloads, however, are often dynamic. Shared cloud systems also present applications with highly dynamic execution environments (multiple users, hardware migration, etc.). It is therefore desirable to continuously re-tune an application during run time (online) in response to changing conditions. Our approach enables online service rate monitoring under most conditions, obviating the need for reliance on steady state predictions for what are probably non-steady state phenomena. First, some of the difficulties associated with online service rate determination are examined. Second, the algorithm to approximate the online non-blocking service rate is described. Lastly, the algorithm is implemented within the open source RaftLib framework for validation using a simple microbenchmark as well as two full streaming applications.