6IMPOSE: Bridging the Reality Gap in 6D Pose Estimation for Robotic Grasping

TL;DR

This paper introduces 6IMPOSE, a comprehensive framework combining synthetic data generation, domain randomization, and optimized inference for robust 6D pose estimation in robotic grasping, achieving high success rates in cluttered, real-world scenarios.

Contribution

The paper presents a novel sim-to-real pipeline with a new dataset, real-time pose estimation, and an integrated robotic grasping system, enhancing generalization and efficiency.

Findings

Achieved 87% success rate in robotic grasping experiments.

Generated large, photo-realistic synthetic RGBD datasets.

Optimized 6D pose estimation for real-time robotic applications.

Abstract

6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE, a novel framework for sim-to-real data generation and 6D pose estimation. 6IMPOSE consists of four modules: First, a data generation pipeline that employs the 3D software suite Blender to create synthetic RGBD image datasets with 6D pose annotations. Second, an annotated RGBD dataset of five household objects generated using the proposed pipeline. Third, a real-time two-stage 6D pose estimation approach that integrates the object detector YOLO-V4 and a streamlined, real-time version of the 6D pose estimation algorithm PVN3D optimized for time-sensitive robotics applications. Fourth, a codebase designed to facilitate the integration of the vision system into a robotic grasping experiment. Our approach demonstrates the efficient generation of large amounts of photo-realistic RGBD images and the successful transfer of the trained inference model to robotic grasping experiments, achieving an overall success rate of 87% in grasping five different household objects from cluttered backgrounds under varying lighting conditions. This is made possible by the fine-tuning of data generation and domain randomization techniques, and the optimization of the inference pipeline, overcoming the generalization and performance shortcomings of the original PVN3D algorithm. Finally, we make the code, synthetic dataset, and all the pretrained models available on Github.