MV6D: Multi-View 6D Pose Estimation on RGB-D Frames Using a Deep Point-wise Voting Network

TL;DR

MV6D is a novel multi-view 6D pose estimation method that fuses RGB-D images from multiple perspectives using a deep point-wise voting network, improving accuracy in cluttered scenes with occlusions.

Contribution

The paper introduces MV6D, an end-to-end trainable multi-view 6D pose estimation approach that does not require multiple prediction stages or fine-tuning, and provides new datasets for cluttered scenes.

Findings

MV6D outperforms state-of-the-art methods in multi-view 6D pose estimation.

MV6D is robust to inaccurate camera poses and dynamic camera setups.

Accuracy improves with more perspectives in the input data.

Abstract

Estimating 6D poses of objects is an essential computer vision task. However, most conventional approaches rely on camera data from a single perspective and therefore suffer from occlusions. We overcome this issue with our novel multi-view 6D pose estimation method called MV6D which accurately predicts the 6D poses of all objects in a cluttered scene based on RGB-D images from multiple perspectives. We base our approach on the PVN3D network that uses a single RGB-D image to predict keypoints of the target objects. We extend this approach by using a combined point cloud from multiple views and fusing the images from each view with a DenseFusion layer. In contrast to current multi-view pose detection networks such as CosyPose, our MV6D can learn the fusion of multiple perspectives in an end-to-end manner and does not require multiple prediction stages or subsequent fine tuning of the prediction. Furthermore, we present three novel photorealistic datasets of cluttered scenes with heavy occlusions. All of them contain RGB-D images from multiple perspectives and the ground truth for instance semantic segmentation and 6D pose estimation. MV6D significantly outperforms the state-of-the-art in multi-view 6D pose estimation even in cases where the camera poses are known inaccurately. Furthermore, we show that our approach is robust towards dynamic camera setups and that its accuracy increases incrementally with an increasing number of perspectives.