A Universal Approach to Feature Representation in Dynamic Task Assignment Problems

TL;DR

This paper introduces a universal graph-based feature representation for dynamic task assignment problems with infinite state and action spaces, enabling effective learning with deep reinforcement learning.

Contribution

It proposes a novel assignment graph representation, maps Petri Nets to these graphs, and adapts PPO for solving complex assignment problems with diverse state and action spaces.

Findings

Effective representation for infinite state/action spaces

Successful learning of near-optimal policies across problem types

Applicable to various archetypal assignment problems

Abstract

Dynamic task assignment concerns the optimal assignment of resources to tasks in a business process. Recently, Deep Reinforcement Learning (DRL) has been proposed as the state of the art for solving assignment problems. DRL methods usually employ a neural network (NN) as an approximator for the policy function, which ingests the state of the process and outputs a valuation of the possible assignments. However, representing the state and the possible assignments so that they can serve as inputs and outputs for a policy NN remains an open challenge, especially when tasks or resources have features with an infinite number of possible values. To solve this problem, this paper proposes a method for representing and solving assignment problems with infinite state and action spaces. In doing so, it provides three contributions: (I) A graph-based feature representation of assignment problems, which we call assignment graph; (II) A mapping from marked Colored Petri Nets to assignment graphs; (III) An adaptation of the Proximal Policy Optimization algorithm that can learn to solve assignment problems represented through assignment graphs. To evaluate the proposed representation method, we model three archetypal assignment problems ranging from finite to infinite state and action space dimensionalities. The experiments show that the method is suitable for representing and learning close-to-optimal task assignment policies regardless of the state and action space dimensionalities.