A Comparison of Deep-Learning Methods for Analysing and Predicting Business Processes

TL;DR

This paper compares various deep-learning models, including GNNs and MLPs, for predicting business process activities and timings, revealing that simple MLPs often outperform more complex models at different process stages.

Contribution

It introduces a comprehensive evaluation of deep-learning models at different process stages, highlighting the effectiveness of simple MLPs over complex models in process prediction tasks.

Findings

MLPs often outperform GNNs and CNNs in prediction accuracy.

Model performance varies significantly across different process stages.

Careful baseline selection is crucial for evaluating process prediction models.

Abstract

Deep-learning models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) have been successfully used for process-mining tasks. They have achieved better performance for different predictive tasks than traditional approaches. We extend the existing body of research by testing four different variants of Graph Neural Networks (GNN) and a fully connected Multi-layer Perceptron (MLP) with dropout for the tasks of predicting the nature and timestamp of the next process activity. In contrast to existing studies, we evaluate our models' performance at different stages of a process, determined by quartiles of the number of events and normalized quarters of the case duration. This provides new insights into the performance of a prediction model, as they behave differently at different stages of a business-process. Interestingly, our experiments show that the simple MLP often outperforms more sophisticated deep-learning models in both prediction tasks. We argue that care needs to be taken when applying automated process-prediction techniques at different stages of a process. We further argue that researchers should reflect their results with strong baselines methods like simple MLPs.