Event Forecasting with Pattern Markov Chains

TL;DR

This paper introduces a probabilistic system for online event forecasting using Pattern Markov Chains, capable of updating predictions in real-time based on streaming data and providing confidence intervals for pattern matches.

Contribution

It presents a novel framework combining Pattern Markov Chains with real-time event streams to produce probabilistic forecasts with guaranteed precision.

Findings

Effective forecasting intervals with high precision

Real-world dataset validation shows accurate predictions

Forecasts are adaptable to streaming data changes

Abstract

We present a system for online probabilistic event forecasting. We assume that a user is interested in detecting and forecasting event patterns, given in the form of regular expressions. Our system can consume streams of events and forecast when the pattern is expected to be fully matched. As more events are consumed, the system revises its forecasts to reflect possible changes in the state of the pattern. The framework of Pattern Markov Chains is used in order to learn a probabilistic model for the pattern, with which forecasts with guaranteed precision may be produced, in the form of intervals within which a full match is expected. Experimental results from real-world datasets are shown and the quality of the produced forecasts is explored, using both precision scores and two other metrics: spread, which refers to the "focusing resolution" of a forecast (interval length), and distance, which captures how early a forecast is reported.