Learning Multi-agent Multi-machine Tending by Mobile Robots

TL;DR

This paper presents a novel multi-agent reinforcement learning framework using mobile robots for flexible and scalable machine tending in manufacturing, outperforming traditional fixed-arm systems.

Contribution

Introduces AB-MAPPO, an attention-based MARL framework for multi-machine tending with mobile robots, enhancing performance over existing MAPPO methods.

Findings

AB-MAPPO outperforms MAPPO in success rate, safety, and resource efficiency.

The attention-based encoding improves learning performance in multi-robot tending tasks.

Extensive ablation studies validate the design choices.

Abstract

Robotics can help address the growing worker shortage challenge of the manufacturing industry. As such, machine tending is a task collaborative robots can tackle that can also highly boost productivity. Nevertheless, existing robotics systems deployed in that sector rely on a fixed single-arm setup, whereas mobile robots can provide more flexibility and scalability. In this work, we introduce a multi-agent multi-machine tending learning framework by mobile robots based on Multi-agent Reinforcement Learning (MARL) techniques with the design of a suitable observation and reward. Moreover, an attention-based encoding mechanism is developed and integrated into Multi-agent Proximal Policy Optimization (MAPPO) algorithm to boost its performance for machine tending scenarios. Our model (AB-MAPPO) outperformed MAPPO in this new challenging scenario in terms of task success, safety, and resources utilization. Furthermore, we provided an extensive ablation study to support our various design decisions.