LLM4PM: A case study on using Large Language Models for Process Modeling in Enterprise Organizations

TL;DR

This paper explores the use of Large Language Models to assist process model creation in organizations, demonstrating a practical application through a chatbot in a multinational company to improve Business Process Management.

Contribution

It presents the first real-world application of LLMs for process modeling, including development, implementation, and evaluation in an industrial setting.

Findings

LLM-based chatbot aids process modellers in creating flow diagrams

Participants found PRODIGY helpful for reducing modeling time

The study provides actionable insights for industry adoption of LLMs

Abstract

We investigate the potential of using Large Language Models (LLM) to support process model creation in organizational contexts. Specifically, we carry out a case study wherein we develop and test an LLM-based chatbot, PRODIGY (PROcess moDellIng Guidance for You), in a multinational company, the Hilti Group. We are particularly interested in understanding how LLM can aid (human) modellers in creating process flow diagrams. To this purpose, we first conduct a preliminary user study (n=10) with professional process modellers from Hilti, inquiring for various pain-points they encounter in their daily routines. Then, we use their responses to design and implement PRODIGY. Finally, we evaluate PRODIGY by letting our user study's participants use PRODIGY, and then ask for their opinion on the pros and cons of PRODIGY. We coalesce our results in actionable takeaways. Through our research, we showcase the first practical application of LLM for process modelling in the real world, shedding light on how industries can leverage LLM to enhance their Business Process Management activities.