The Impact of Overall Optimization on Warehouse Automation

TL;DR

This paper introduces a centralized training and decentralized execution MARL framework for optimizing robot coordination in automated warehouses, demonstrating improved overall performance over partial optimization methods.

Contribution

The paper proposes a novel MARL framework with a shared critic for heterogeneous agents, enhancing overall warehouse automation optimization.

Findings

The proposed MARL framework outperforms rule-based methods in task selection.

Overall optimization significantly improves warehouse efficiency.

The shared critic approach effectively manages asynchronous decision-making.

Abstract

In this study, we propose a novel approach for investigating optimization performance by flexible robot coordination in automated warehouses with multi-agent reinforcement learning (MARL)-based control. Automated systems using robots are expected to achieve efficient operations compared with manual systems in terms of overall optimization performance. However, the impact of overall optimization on performance remains unclear in most automated systems due to a lack of suitable control methods. Thus, we proposed a centralized training-and-decentralized execution MARL framework as a practical overall optimization control method. In the proposed framework, we also proposed a single shared critic, trained with global states and rewards, applicable to a case in which heterogeneous agents make decisions asynchronously. Our proposed MARL framework was applied to the task selection of material handling equipment through automated order picking simulation, and its performance was evaluated to determine how far overall optimization outperforms partial optimization by comparing it with other MARL frameworks and rule-based control methods.