A Rollout-Based Algorithm and Reward Function for Resource Allocation in Business Processes

TL;DR

This paper introduces a rollout-based deep reinforcement learning algorithm with a directly decomposed reward function to optimize resource allocation in business processes, effectively minimizing cycle time and adapting to dynamic environments.

Contribution

It presents a novel rollout-based DRL algorithm and a reward function that directly aligns with the objective, eliminating the need for reward engineering in resource allocation tasks.

Findings

The algorithm learns the optimal policy in test scenarios.

It outperforms or matches existing heuristics on complex process models.

The method effectively minimizes cycle time in diverse business process scenarios.

Abstract

Resource allocation plays a critical role in minimizing cycle time and improving the efficiency of business processes. Recently, Deep Reinforcement Learning (DRL) has emerged as a powerful technique to optimize resource allocation policies in business processes. In the DRL framework, an agent learns a policy through interaction with the environment, guided solely by reward signals that indicate the quality of its decisions. However, existing algorithms are not suitable for dynamic environments such as business processes. Furthermore, existing DRL-based methods rely on engineered reward functions that approximate the desired objective, but a misalignment between reward and objective can lead to undesired decisions or suboptimal policies. To address these issues, we propose a rollout-based DRL algorithm and a reward function to optimize the objective directly. Our algorithm iteratively improves the policy by evaluating execution trajectories following different actions. Our reward function directly decomposes the objective function of minimizing the cycle time, such that trial-and-error reward engineering becomes unnecessary. We evaluated our method in six scenarios, for which the optimal policy can be computed, and on a set of increasingly complex, realistically sized process models. The results show that our algorithm can learn the optimal policy for the scenarios and outperform or match the best heuristics on the realistically sized business processes.