Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environment

TL;DR

This paper introduces a hierarchical deep reinforcement learning approach for dynamic task scheduling and execution in warehouse multi-agent environments, addressing challenges of large action spaces and partial observability.

Contribution

It proposes a novel hierarchical RL framework for centralized scheduling and decentralized execution, including scenarios with no central coordination at test time.

Findings

Hierarchical RL effectively manages dynamic task scheduling.

Decentralized agents learn to cooperate with partial observability.

The approach adapts to environments lacking central control.

Abstract

In a warehouse environment, tasks appear dynamically. Consequently, a task management system that matches them with the workforce too early (e.g., weeks in advance) is necessarily sub-optimal. Also, the rapidly increasing size of the action space of such a system consists of a significant problem for traditional schedulers. Reinforcement learning, however, is suited to deal with issues requiring making sequential decisions towards a long-term, often remote, goal. In this work, we set ourselves on a problem that presents itself with a hierarchical structure: the task-scheduling, by a centralised agent, in a dynamic warehouse multi-agent environment and the execution of one such schedule, by decentralised agents with only partial observability thereof. We propose to use deep reinforcement learning to solve both the high-level scheduling problem and the low-level multi-agent problem of schedule execution. Finally, we also conceive the case where centralisation is impossible at test time and workers must learn how to cooperate in executing the tasks in an environment with no schedule and only partial observability.