Can deep neural networks learn process model structure? An assessment framework and analysis

TL;DR

This paper evaluates whether deep neural networks, especially RNNs like LSTMs, can learn the structural aspects of process models, introducing new metrics and an assessment framework for this purpose.

Contribution

It proposes a novel evaluation framework with tailored metrics to measure neural networks' ability to learn process model structures from data.

Findings

Neural networks require careful tuning to learn process structures.

Even simple process models pose challenges for neural network learning.

The framework enables systematic assessment of process model learning capabilities.

Abstract

Predictive process monitoring concerns itself with the prediction of ongoing cases in (business) processes. Prediction tasks typically focus on remaining time, outcome, next event or full case suffix prediction. Various methods using machine and deep learning havebeen proposed for these tasks in recent years. Especially recurrent neural networks (RNNs) such as long short-term memory nets (LSTMs) have gained in popularity. However, no research focuses on whether such neural network-based models can truly learn the structure of underlying process models. For instance, can such neural networks effectively learn parallel behaviour or loops? Therefore, in this work, we propose an evaluation scheme complemented with new fitness, precision, and generalisation metrics, specifically tailored towards measuring the capacity of deep learning models to learn process model structure. We apply this framework to several process models with simple control-flow behaviour, on the task of next-event prediction. Our results show that, even for such simplistic models, careful tuning of overfitting countermeasures is required to allow these models to learn process model structure.