Handling out-of-order input arrival in CEP engines on the edge combining optimistic, pessimistic and lazy evaluation

TL;DR

LimeCEP is a hybrid Complex Event Processing system that efficiently manages out-of-order, late, and duplicate events on resource-constrained devices by combining lazy, optimistic, and buffering strategies, significantly improving performance.

Contribution

It introduces LimeCEP, a novel hybrid approach integrating multiple evaluation strategies and Kafka for improved handling of data inconsistencies in CEP on edge devices.

Findings

Up to six orders of magnitude lower latency compared to state-of-the-art

Ten times lower memory usage

Six times lower CPU utilization

Abstract

In Complex Event Processing, handling out-of-order, late, and duplicate events is critical for real-time analytics, especially on resource-constrained devices that process heterogeneous data from multiple sources. We present LimeCEP, a hybrid CEP approach that combines lazy evaluation, buffering, and speculative processing to efficiently handle data inconsistencies while supporting multi-pattern detection under relaxed semantics. LimeCEP integrates Kafka for efficient message ordering, retention, and duplicate elimination, and offers configurable strategies to trade off between accuracy, latency, and resource consumption. Compared to state-of-the-art systems like SASE and FlinkCEP, LimeCEP achieves up to six orders of magnitude lower latency, with up to 10 times lower memory usage and 6 times lower CPU utilization, while maintaining near-perfect precision and recall under high-disorder input streams, making it well-suited for non-cloud deployments.