Unsupervised Frequent Pattern Mining for CEP

TL;DR

REDEEMER is a novel reinforcement and active learning system that automatically mines complex event processing patterns from data streams, reducing human effort and enabling CEP in new domains.

Contribution

It introduces a reinforcement learning approach with a new policy gradient method and combines active learning to efficiently discover new CEP rules with minimal labeled data.

Findings

REDEEMER outperforms existing reinforcement learning methods in pattern mining.

It successfully extends CEP pattern knowledge in diverse datasets.

REDEEMER can suggest previously unobserved CEP rules.

Abstract

Complex Event Processing (CEP) is a set of methods that allow efficient knowledge extraction from massive data streams using complex and highly descriptive patterns. Numerous applications, such as online finance, healthcare monitoring and fraud detection use CEP technologies to capture critical alerts, potential threats, or vital notifications in real time. As of today, in many fields, patterns are manually defined by human experts. However, desired patterns often contain convoluted relations that are difficult for humans to detect, and human expertise is scarce in many domains. We present REDEEMER (REinforcement baseD cEp pattErn MinER), a novel reinforcement and active learning approach aimed at mining CEP patterns that allow expansion of the knowledge extracted while reducing the human effort required. This approach includes a novel policy gradient method for vast multivariate spaces and a new way to combine reinforcement and active learning for CEP rule learning while minimizing the number of labels needed for training. REDEEMER aims to enable CEP integration in domains that could not utilize it before. To the best of our knowledge, REDEEMER is the first system that suggests new CEP rules that were not observed beforehand, and is the first method aimed for increasing pattern knowledge in fields where experts do not possess sufficient information required for CEP tools. Our experiments on diverse data-sets demonstrate that REDEEMER is able to extend pattern knowledge while outperforming several state-of-the-art reinforcement learning methods for pattern mining.