Toward a Plug-and-Play Vision-Based Grasping Module for Robotics

TL;DR

This paper presents a versatile, vision-based grasping framework for robots that uses Quality-Diversity algorithms to generate diverse grasp trajectories, improving adaptability and transferability across different robotic manipulators.

Contribution

The work introduces a modular, transferable grasping system combining multiple vision modules and QD algorithms, addressing reproducibility and generalization issues in robotic grasping.

Findings

Achieved comparable grasping performance on different robot arms.

Demonstrated effective transfer of grasping trajectories from simulation to real robots.

Enhanced adaptability of grasping strategies without additional training.

Abstract

Despite recent advancements in AI for robotics, grasping remains a partially solved challenge, hindered by the lack of benchmarks and reproducibility constraints. This paper introduces a vision-based grasping framework that can easily be transferred across multiple manipulators. Leveraging Quality-Diversity (QD) algorithms, the framework generates diverse repertoires of open-loop grasping trajectories, enhancing adaptability while maintaining a diversity of grasps. This framework addresses two main issues: the lack of an off-the-shelf vision module for detecting object pose and the generalization of QD trajectories to the whole robot operational space. The proposed solution combines multiple vision modules for 6DoF object detection and tracking while rigidly transforming QD-generated trajectories into the object frame. Experiments on a Franka Research 3 arm and a UR5 arm with a SIH Schunk hand demonstrate comparable performance when the real scene aligns with the simulation used for grasp generation. This work represents a significant stride toward building a reliable vision-based grasping module transferable to new platforms, while being adaptable to diverse scenarios without further training iterations.