Object-RPE: Dense 3D Reconstruction and Pose Estimation with Convolutional Neural Networks for Warehouse Robots

TL;DR

This paper introduces Object-RPE, a system combining CNNs and dense SLAM to achieve dense 3D scene reconstruction and accurate 6D object pose estimation in warehouse environments, improving robustness and accuracy over existing methods.

Contribution

It presents a novel framework integrating CNNs with dense SLAM for multi-view 6D pose estimation and semantic reconstruction in large environments, outperforming prior single-view approaches.

Findings

Enhanced 3D reconstruction quality demonstrated on datasets.

Improved 6D pose estimation accuracy over state-of-the-art.

Effective multi-view fusion increases robustness of object detection.

Abstract

We present an approach for recognizing all objects in a scene and estimating their full pose from an accurate 3D instance-aware semantic reconstruction using an RGB-D camera. Our framework couples convolutional neural networks (CNNs) and a state-of-the-art dense Simultaneous Localisation and Mapping (SLAM) system, ElasticFusion, to achieve both high-quality semantic reconstruction as well as robust 6D pose estimation for relevant objects. While the main trend in CNN-based 6D pose estimation has been to infer object's position and orientation from single views of the scene, our approach explores performing pose estimation from multiple viewpoints, under the conjecture that combining multiple predictions can improve the robustness of an object detection system. The resulting system is capable of producing high-quality object-aware semantic reconstructions of room-sized environments, as well as accurately detecting objects and their 6D poses. The developed method has been verified through experimental validation on the YCB-Video dataset and a newly collected warehouse object dataset. Experimental results confirmed that the proposed system achieves improvements over state-of-the-art methods in terms of surface reconstruction and object pose prediction. Our code and video are available at https://sites.google.com/view/object-rpe.