Bridging the Reality Gap for Pose Estimation Networks using Sensor-Based Domain Randomization

TL;DR

This paper presents a novel sensor-based domain randomization technique for pose estimation networks that effectively bridges the performance gap between synthetic and real data by integrating 3D data and introducing the SparseEdge feature.

Contribution

It introduces a new method that incorporates 3D data into pose estimation networks using sensor-based augmentation and the SparseEdge feature to improve accuracy with synthetic training data.

Findings

Outperforms previous synthetic-data-trained methods on benchmarks.

Achieves results comparable to real-data-trained methods.

Enhances pose estimation accuracy through 3D data integration.

Abstract

Since the introduction of modern deep learning methods for object pose estimation, test accuracy and efficiency has increased significantly. For training, however, large amounts of annotated training data are required for good performance. While the use of synthetic training data prevents the need for manual annotation, there is currently a large performance gap between methods trained on real and synthetic data. This paper introduces a new method, which bridges this gap. Most methods trained on synthetic data use 2D images, as domain randomization in 2D is more developed. To obtain precise poses, many of these methods perform a final refinement using 3D data. Our method integrates the 3D data into the network to increase the accuracy of the pose estimation. To allow for domain randomization in 3D, a sensor-based data augmentation has been developed. Additionally, we introduce the SparseEdge feature, which uses a wider search space during point cloud propagation to avoid relying on specific features without increasing run-time. Experiments on three large pose estimation benchmarks show that the presented method outperforms previous methods trained on synthetic data and achieves comparable results to existing methods trained on real data.