Learning Manipulation Tasks in Dynamic and Shared 3D Spaces

TL;DR

This paper introduces a deep reinforcement learning approach for multi-categorical item placement in shared 3D spaces with dual manipulators, improving automation in material handling tasks involving humans and robots.

Contribution

It presents a novel deep RL strategy using an actor-critic framework in a dynamic 3D environment for collaborative multi-robot pick-and-place tasks.

Findings

Increased cumulative reward for agents farther from human factors

Effective learning in environments with static and dynamic obstacles

Potential for improved automation in shared workspaces

Abstract

Automating the segregation process is a need for every sector experiencing a high volume of materials handling, repetitive and exhaustive operations, in addition to risky exposures. Learning automated pick-and-place operations can be efficiently done by introducing collaborative autonomous systems (e.g. manipulators) in the workplace and among human operators. In this paper, we propose a deep reinforcement learning strategy to learn the place task of multi-categorical items from a shared workspace between dual-manipulators and to multi-goal destinations, assuming the pick has been already completed. The learning strategy leverages first a stochastic actor-critic framework to train an agent's policy network, and second, a dynamic 3D Gym environment where both static and dynamic obstacles (e.g. human factors and robot mate) constitute the state space of a Markov decision process. Learning is conducted in a Gazebo simulator and experiments show an increase in cumulative reward function for the agent further away from human factors. Future investigations will be conducted to enhance the task performance for both agents simultaneously.