Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study

TL;DR

This paper explores how large language models can be used for process mining by abstracting process artifacts and designing effective prompts, demonstrated through a case study with real event logs.

Contribution

It introduces abstraction techniques and prompting strategies for process mining with LLMs, integrated into an open-source library, and evaluates their effectiveness in a case study.

Findings

LLMs can effectively answer process mining queries with proper abstractions.

Prompt design significantly impacts the quality of LLM responses.

The approach is feasible for analyzing real-world event logs.

Abstract

Large Language Models (LLMs) are capable of answering questions in natural language for various purposes. With recent advancements (such as GPT-4), LLMs perform at a level comparable to humans for many proficient tasks. The analysis of business processes could benefit from a natural process querying language and using the domain knowledge on which LLMs have been trained. However, it is impossible to provide a complete database or event log as an input prompt due to size constraints. In this paper, we apply LLMs in the context of process mining by i) abstracting the information of standard process mining artifacts and ii) describing the prompting strategies. We implement the proposed abstraction techniques into pm4py, an open-source process mining library. We present a case study using available event logs. Starting from different abstractions and analysis questions, we formulate prompts and evaluate the quality of the answers.