Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

TL;DR

This paper presents a deep neural network trained solely on synthetic data that accurately estimates 6-DoF object poses from a single RGB image, enabling effective real-time robotic grasping in household environments.

Contribution

It introduces a novel synthetic data generation approach combining domain randomization and photorealism, enabling state-of-the-art pose estimation without real training data.

Findings

Achieves competitive performance with networks trained on real data

Generalizes well to novel environments and lighting conditions

Enables real-time robotic grasping of household objects

Abstract

Using synthetic data for training deep neural networks for robotic manipulation holds the promise of an almost unlimited amount of pre-labeled training data, generated safely out of harm's way. One of the key challenges of synthetic data, to date, has been to bridge the so-called reality gap, so that networks trained on synthetic data operate correctly when exposed to real-world data. We explore the reality gap in the context of 6-DoF pose estimation of known objects from a single RGB image. We show that for this problem the reality gap can be successfully spanned by a simple combination of domain randomized and photorealistic data. Using synthetic data generated in this manner, we introduce a one-shot deep neural network that is able to perform competitively against a state-of-the-art network trained on a combination of real and synthetic data. To our knowledge, this is the first deep network trained only on synthetic data that is able to achieve state-of-the-art performance on 6-DoF object pose estimation. Our network also generalizes better to novel environments including extreme lighting conditions, for which we show qualitative results. Using this network we demonstrate a real-time system estimating object poses with sufficient accuracy for real-world semantic grasping of known household objects in clutter by a real robot.