SyMFM6D: Symmetry-aware Multi-directional Fusion for Multi-View 6D Object Pose Estimation

TL;DR

SyMFM6D introduces a multi-view, symmetry-aware deep learning approach for 6D object pose estimation that effectively handles occlusions, ambiguities, and symmetric object challenges, outperforming existing methods.

Contribution

The paper presents a novel multi-view fusion network with symmetry-aware training for 6D pose estimation, addressing symmetry ambiguities and robustness issues.

Findings

Outperforms state-of-the-art in single-view and multi-view 6D pose estimation

Effective symmetry-aware training improves accuracy for symmetric objects

Robust to camera calibration errors and dynamic setups

Abstract

Detecting objects and estimating their 6D poses is essential for automated systems to interact safely with the environment. Most 6D pose estimators, however, rely on a single camera frame and suffer from occlusions and ambiguities due to object symmetries. We overcome this issue by presenting a novel symmetry-aware multi-view 6D pose estimator called SyMFM6D. Our approach efficiently fuses the RGB-D frames from multiple perspectives in a deep multi-directional fusion network and predicts predefined keypoints for all objects in the scene simultaneously. Based on the keypoints and an instance semantic segmentation, we efficiently compute the 6D poses by least-squares fitting. To address the ambiguity issues for symmetric objects, we propose a novel training procedure for symmetry-aware keypoint detection including a new objective function. Our SyMFM6D network significantly outperforms the state-of-the-art in both single-view and multi-view 6D pose estimation. We furthermore show the effectiveness of our symmetry-aware training procedure and demonstrate that our approach is robust towards inaccurate camera calibration and dynamic camera setups.