Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets

TL;DR

This paper explores how automatically generated grasping datasets using domain randomization can improve sim-to-real transfer in robotic grasping, achieving high transfer ratios and identifying key challenges in bridging the reality gap.

Contribution

It introduces a QD-based method to enhance grasp robustness under domain randomization, with extensive real-world testing across multiple robotic arms.

Findings

Over 7000 grasp trajectories generated and tested in real world

Correlation identified between quality criteria and transferability

Achieved 84% transfer ratio on Franka Research 3 arm

Abstract

Robotic grasping refers to making a robotic system pick an object by applying forces and torques on its surface. Many recent studies use data-driven approaches to address grasping, but the sparse reward nature of this task made the learning process challenging to bootstrap. To avoid constraining the operational space, an increasing number of works propose grasping datasets to learn from. But most of them are limited to simulations. The present paper investigates how automatically generated grasps can be exploited in the real world. More than 7000 reach-and-grasp trajectories have been generated with Quality-Diversity (QD) methods on 3 different arms and grippers, including parallel fingers and a dexterous hand, and tested in the real world. Conducted analysis on the collected measure shows correlations between several Domain Randomization-based quality criteria and sim-to-real transferability. Key challenges regarding the reality gap for grasping have been identified, stressing matters on which researchers on grasping should focus in the future. A QD approach has finally been proposed for making grasps more robust to domain randomization, resulting in a transfer ratio of 84% on the Franka Research 3 arm.