A Reinforcement Learning Approach for Robotic Unloading from Visual Observations

TL;DR

This paper presents a hierarchical reinforcement learning framework for robotic unloading from visual data, emphasizing sample efficiency and safety, with experimental validation and open-source code for benchmarking.

Contribution

It introduces a novel hierarchical DRL-based controller with safety mechanisms for unloading tasks, reducing reliance on labeled data and enhancing learning performance.

Findings

Hierarchical DRL improves unloading success rates.

Safety bias enhances learning stability.

Open-source code facilitates future research.

Abstract

In this work, we focus on a robotic unloading problem from visual observations, where robots are required to autonomously unload stacks of parcels using RGB-D images as their primary input source. While supervised and imitation learning have accomplished good results in these types of tasks, they heavily rely on labeled data, which are challenging to obtain in realistic scenarios. Our study aims to develop a sample efficient controller framework that can learn unloading tasks without the need for labeled data during the learning process. To tackle this challenge, we propose a hierarchical controller structure that combines a high-level decision-making module with classical motion control. The high-level module is trained using Deep Reinforcement Learning (DRL), wherein we incorporate a safety bias mechanism and design a reward function tailored to this task. Our experiments demonstrate that both these elements play a crucial role in achieving improved learning performance. Furthermore, to ensure reproducibility and establish a benchmark for future research, we provide free access to our code and simulation.