Improved Reinforcement Learning Pushing Policies via Heuristic Rules

TL;DR

This paper introduces a heuristic rule integrated into reinforcement learning to improve robotic pushing policies for object singulation, resulting in more efficient training and decision-making that mimics the heuristic.

Contribution

The paper presents a novel heuristic rule for object singulation and demonstrates its integration into RL training to enhance performance and learning efficiency.

Findings

Heuristic rule improves RL singulation performance.

RL policies implicitly learn heuristic-like decision strategies.

Simulation results show increased efficiency and effectiveness.

Abstract

Non-prehensile pushing actions have the potential to singulate a target object from its surrounding clutter in order to facilitate the robotic grasping of the target. To address this problem we utilize a heuristic rule that moves the target object towards the workspace's empty space and demonstrate that this simple heuristic rule achieves singulation. We incorporate this effective heuristic rule to the reward in order to train more efficiently reinforcement learning (RL) agents for singulation. Simulation experiments demonstrate that this insight increases performance. Finally, our results show that the RL-based policy implicitly learns something similar to one of the used heuristics in terms of decision making. Qualitative results, code, pre-trained models and simulation environments are available at https://github.com/robot-clutter/improved_rl.