Unsupervised Reward Shaping for a Robotic Sequential Picking Task from Visual Observations in a Logistics Scenario

TL;DR

This paper introduces an unsupervised reward shaping method for robotic pick-and-place tasks in logistics, enhancing reinforcement learning efficiency without requiring extensive supervision.

Contribution

The paper proposes a novel unsupervised reward shaping algorithm based on expert observations, improving RL performance in sequential robotic tasks.

Findings

Enhanced RL performance in logistics tasks

Reduced supervision requirements

Theoretically motivated reward shaping approach

Abstract

We focus on an unloading problem, typical of the logistics sector, modeled as a sequential pick-and-place task. In this type of task, modern machine learning techniques have shown to work better than classic systems since they are more adaptable to stochasticity and better able to cope with large uncertainties. More specifically, supervised and imitation learning have achieved outstanding results in this regard, with the shortcoming of requiring some form of supervision which is not always obtainable for all settings. On the other hand, reinforcement learning (RL) requires much milder form of supervision but still remains impracticable due to its inefficiency. In this paper, we propose and theoretically motivate a novel Unsupervised Reward Shaping algorithm from expert's observations which relaxes the level of supervision required by the agent and works on improving RL performance in our task.