Recommending the optimal policy by learning to act from temporal data

TL;DR

This paper introduces a reinforcement learning-based method to recommend optimal process actions from execution logs, aiming to optimize key performance indicators without relying on explicit process models.

Contribution

It presents a novel approach that learns an optimal policy directly from temporal execution data using reinforcement learning, outperforming some deep RL methods.

Findings

Effective policy learning from real and synthetic logs

Outperforms off-policy Deep RL approaches in experiments

Demonstrates the potential of white box RL in process monitoring

Abstract

Prescriptive Process Monitoring is a prominent problem in Process Mining, which consists in identifying a set of actions to be recommended with the goal of optimising a target measure of interest or Key Performance Indicator (KPI). One challenge that makes this problem difficult is the need to provide Prescriptive Process Monitoring techniques only based on temporally annotated (process) execution data, stored in, so-called execution logs, due to the lack of well crafted and human validated explicit models. In this paper we aim at proposing an AI based approach that learns, by means of Reinforcement Learning (RL), an optimal policy (almost) only from the observation of past executions and recommends the best activities to carry on for optimizing a KPI of interest. This is achieved first by learning a Markov Decision Process for the specific KPIs from data, and then by using RL training to learn the optimal policy. The approach is validated on real and synthetic datasets and compared with off-policy Deep RL approaches. The ability of our approach to compare with, and often overcome, Deep RL approaches provides a contribution towards the exploitation of white box RL techniques in scenarios where only temporal execution data are available.