Fast-Learning Grasping and Pre-Grasping via Clutter Quantization and Q-map Masking

TL;DR

This paper introduces a fast-learning framework for robotic grasping in cluttered environments that combines pre-grasp actions, clutter quantization, and Q-map masking, achieving high success rates with minimal real-world training.

Contribution

The proposed FLG framework integrates pre-grasp actions with grasping using deep reinforcement learning, clutter quantization, and Q-map masking for efficient training and improved accuracy.

Findings

94% grasp success rate in cluttered scenarios

Effective transfer from simulation to real-world with minimal fine-tuning

Achieves comparable or better performance than state-of-the-art methods

Abstract

Grasping objects in cluttered scenarios is a challenging task in robotics. Performing pre-grasp actions such as pushing and shifting to scatter objects is a way to reduce clutter. Based on deep reinforcement learning, we propose a Fast-Learning Grasping (FLG) framework, that can integrate pre-grasping actions along with grasping to pick up objects from cluttered scenarios with reduced real-world training time. We associate rewards for performing moving actions with the change of environmental clutter and utilize a hybrid triggering method, leading to data-efficient learning and synergy. Then we use the output of an extended fully convolutional network as the value function of each pixel point of the workspace and establish an accurate estimation of the grasp probability for each action. We also introduce a mask function as prior knowledge to enable the agents to focus on the accurate pose adjustment to improve the effectiveness of collecting training data and, hence, to learn efficiently. We carry out pre-training of the FLG over simulated environment, and then the learnt model is transferred to the real world with minimal fine-tuning for further learning during actions. Experimental results demonstrate a 94% grasp success rate and the ability to generalize to novel objects. Compared to state-of-the-art approaches in the literature, the proposed FLG framework can achieve similar or higher grasp success rate with lesser amount of training in the real world. Supplementary video is available at https://youtu.be/e04uDLsxfDg.