ESPBench: The Enterprise Stream Processing Benchmark

TL;DR

ESPBench is a comprehensive benchmark designed for enterprise stream processing systems, addressing the lack of suitable evaluation tools for complex, real-world data streaming applications in Industry 4.0 and IoT contexts.

Contribution

It introduces a new benchmark architecture, query workload, and validation toolkit specifically for enterprise stream processing, tested on leading systems like Spark, Flink, and Hazelcast Jet.

Findings

ESPBench enables objective latency measurement.

The toolkit supports query result validation.

Benchmark results highlight system performance differences.

Abstract

Growing data volumes and velocities in fields such as Industry 4.0 or the Internet of Things have led to the increased popularity of data stream processing systems. Enterprises can leverage these developments by enriching their core business data and analyses with up-to-date streaming data. Comparing streaming architectures for these complex use cases is challenging, as existing benchmarks do not cover them. ESPBench is a new enterprise stream processing benchmark that fills this gap. We present its architecture, the benchmarking process, and the query workload. We employ ESPBench on three state-of-the-art stream processing systems, Apache Spark, Apache Flink, and Hazelcast Jet, using provided query implementations developed with Apache Beam. Our results highlight the need for the provided ESPBench toolkit that supports benchmark execution, as it enables query result validation and objective latency measures.