LMStream: When Distributed Micro-Batch Stream Processing Systems Meet GPU

TL;DR

LMStream is a GPU-accelerated micro-batch streaming system that dynamically optimizes batching and query planning to achieve bounded latency and high throughput in real-time data processing.

Contribution

It introduces novel dynamic batching and query planning mechanisms for micro-batch stream processing on GPUs, ensuring bounded latency and improved throughput.

Findings

Latency improved up to 70.7%

Throughput increased by up to 1.74x

Supports real-time online parameter optimization

Abstract

This paper presents LMStream, which ensures bounded latency while maximizing the throughput on the GPU-enabled micro-batch streaming systems. The main ideas behind LMStream's design can be summarized as two novel mechanisms: (1) dynamic batching and (2) dynamic operation-level query planning. By controlling the micro-batch size, LMStream significantly reduces the latency of individual dataset because it does not perform unconditional buffering only for improving GPU utilization. LMStream bounds the latency to an optimal value according to the characteristics of the window operation used in the streaming application. Dynamic mapping between a query to an execution device based on the data size and dynamic device preference improves both the throughput and latency as much as possible. In addition, LMStream proposes a low-overhead online cost model parameter optimization method without interrupting the real-time stream processing. We implemented LMStream on Apache Spark, which supports micro-batch stream processing. Compared to the previous throughput-oriented method, LMStream showed an average latency improvement up to a maximum of 70.7%, while improving average throughput up to 1.74x.