SPECTRE: Supporting Consumption Policies in Window-Based Parallel Complex Event Processing

TL;DR

SPECTRE introduces a speculative framework for parallel processing of dependent windows in distributed complex event processing, enabling high throughput despite consumption policies that restrict event reuse across windows.

Contribution

It proposes a novel speculative approach to handle consumption policies in window-based parallel CEP, improving scalability and throughput.

Findings

Achieves up to linear scalability with CPU cores.

Effectively manages event consumption dependencies.

Enhances parallel processing in complex event detection.

Abstract

Distributed Complex Event Processing (DCEP) is a paradigm to infer the occurrence of complex situations in the surrounding world from basic events like sensor readings. In doing so, DCEP operators detect event patterns on their incoming event streams. To yield high operator throughput, data parallelization frameworks divide the incoming event streams of an operator into overlapping windows that are processed in parallel by a number of operator instances. In doing so, the basic assumption is that the different windows can be processed independently from each other. However, consumption policies enforce that events can only be part of one pattern instance; then, they are consumed, i.e., removed from further pattern detection. That implies that the constituent events of a pattern instance detected in one window are excluded from all other windows as well, which breaks the data parallelism between different windows. In this paper, we tackle this problem by means of speculation: Based on the likelihood of an event's consumption in a window, subsequent windows may speculatively suppress that event. We propose the SPECTRE framework for speculative processing of multiple dependent windows in parallel. Our evaluations show an up to linear scalability of SPECTRE with the number of CPU cores.