Reinforcement Learning for Location-Aware Scheduling

TL;DR

This paper explores how deep reinforcement learning can optimize location-aware scheduling in warehouse environments, addressing challenges of large state spaces through compact representations and analyzing environment factors affecting performance.

Contribution

It introduces a compact state-action representation for multi-agent warehouse scheduling and evaluates how environment complexity impacts learning and transferability.

Findings

Performance varies with floor plan complexity and agent location information.

Agents trained in specific environments can generalize to unseen settings.

Floor plan geometry correlates with performance degradation.

Abstract

Recent techniques in dynamical scheduling and resource management have found applications in warehouse environments due to their ability to organize and prioritize tasks in a higher temporal resolution. The rise of deep reinforcement learning, as a learning paradigm, has enabled decentralized agent populations to discover complex coordination strategies. However, training multiple agents simultaneously introduce many obstacles in training as observation and action spaces become exponentially large. In our work, we experimentally quantify how various aspects of the warehouse environment (e.g., floor plan complexity, information about agents' live location, level of task parallelizability) affect performance and execution priority. To achieve efficiency, we propose a compact representation of the state and action space for location-aware multi-agent systems, wherein each agent has knowledge of only self and task coordinates, hence only partial observability of the underlying Markov Decision Process. Finally, we show how agents trained in certain environments maintain performance in completely unseen settings and also correlate performance degradation with floor plan geometry.