Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs

TL;DR

This paper extends a framework using large language models for predictive process monitoring, demonstrating their effectiveness in small data scenarios and revealing their reasoning capabilities across multiple KPIs.

Contribution

It introduces a comprehensive evaluation of LLMs in predictive process monitoring, highlighting their ability to leverage prior knowledge and perform higher-order reasoning.

Findings

LLMs outperform benchmarks in data-scarce settings with 100 traces.

LLMs utilize internal correlations and prior knowledge for predictions.

The model employs higher-order reasoning rather than simple pattern replication.

Abstract

Predictive Process Monitoring is a branch of process mining that aims to predict the outcome of an ongoing process. Recently, it leveraged machine-and-deep learning architectures. In this paper, we extend our prior LLM-based Predictive Process Monitoring framework, which was initially focused on total time prediction via prompting. The extension consists of comprehensively evaluating its generality, semantic leverage, and reasoning mechanisms, also across multiple Key Performance Indicators. Empirical evaluations conducted on three distinct event logs and across the Key Performance Indicators of Total Time and Activity Occurrence prediction indicate that, in data-scarce settings with only 100 traces, the LLM surpasses the benchmark methods. Furthermore, the experiments also show that the LLM exploits both its embodied prior knowledge and the internal correlations among training traces. Finally, we examine the reasoning strategies employed by the model, demonstrating that the LLM does not merely replicate existing predictive methods but performs higher-order reasoning to generate the predictions.