PEGG-Net: Pixel-Wise Efficient Grasp Generation in Complex Scenes

TL;DR

PEGG-Net is a novel pixel-wise grasp generation network that significantly improves grasping accuracy in complex, cluttered scenes and can operate in dynamic environments, advancing robotic manipulation capabilities.

Contribution

The paper introduces PEGG-Net, a new pixel-wise grasp estimation network that outperforms existing methods in complex scenes without complex structures.

Findings

Achieves 98.9% accuracy on Cornell dataset

Attains 93.8% on Jacquard dataset

Demonstrates high success rate in real-world cluttered and dynamic scenes

Abstract

Vision-based grasp estimation is an essential part of robotic manipulation tasks in the real world. Existing planar grasp estimation algorithms have been demonstrated to work well in relatively simple scenes. But when it comes to complex scenes, such as cluttered scenes with messy backgrounds and moving objects, the algorithms from previous works are prone to generate inaccurate and unstable grasping contact points. In this work, we first study the existing planar grasp estimation algorithms and analyze the related challenges in complex scenes. Secondly, we design a Pixel-wise Efficient Grasp Generation Network (PEGG-Net) to tackle the problem of grasping in complex scenes. PEGG-Net can achieve improved state-of-the-art performance on the Cornell dataset (98.9%) and second-best performance on the Jacquard dataset (93.8%), outperforming other existing algorithms without the introduction of complex structures. Thirdly, PEGG-Net could operate in a closed-loop manner for added robustness in dynamic environments using position-based visual servoing (PBVS). Finally, we conduct real-world experiments on static, dynamic, and cluttered objects in different complex scenes. The results show that our proposed network achieves a high success rate in grasping irregular objects, household objects, and workshop tools. To benefit the community, our trained model and supplementary materials are available at https://github.com/HZWang96/PEGG-Net.