ProcessTransformer: Predictive Business Process Monitoring with Transformer Network

TL;DR

ProcessTransformer leverages transformer networks with self-attention to improve predictive accuracy in business process monitoring, especially for long sequences, outperforming prior methods in next activity prediction.

Contribution

The paper introduces ProcessTransformer, a novel transformer-based model that captures long-range dependencies in event logs for predictive process monitoring tasks.

Findings

Achieves over 80% accuracy in next activity prediction

Outperforms baseline models in multiple predictive tasks

Effective on nine real-world event logs

Abstract

Predictive business process monitoring focuses on predicting future characteristics of a running process using event logs. The foresight into process execution promises great potentials for efficient operations, better resource management, and effective customer services. Deep learning-based approaches have been widely adopted in process mining to address the limitations of classical algorithms for solving multiple problems, especially the next event and remaining-time prediction tasks. Nevertheless, designing a deep neural architecture that performs competitively across various tasks is challenging as existing methods fail to capture long-range dependencies in the input sequences and perform poorly for lengthy process traces. In this paper, we propose ProcessTransformer, an approach for learning high-level representations from event logs with an attention-based network. Our model incorporates long-range memory and relies on a self-attention mechanism to establish dependencies between a multitude of event sequences and corresponding outputs. We evaluate the applicability of our technique on nine real event logs. We demonstrate that the transformer-based model outperforms several baselines of prior techniques by obtaining on average above 80% accuracy for the task of predicting the next activity. Our method also perform competitively, compared to baselines, for the tasks of predicting event time and remaining time of a running case