MaskVal: Simple but Effective Uncertainty Quantification for 6D Pose Estimation

TL;DR

MaskVal offers a simple yet effective method for uncertainty quantification in 6D pose estimation, improving safety and reliability in robotic applications without modifying existing pose estimators.

Contribution

The paper introduces MaskVal, a novel uncertainty quantification approach that compares pose estimates with segmentations via rendering, outperforming ensemble methods without altering pose estimators.

Findings

MaskVal significantly outperforms ensemble methods in uncertainty estimation.

Using MaskVal improves the safety and reliability of 6D pose estimators in robotic setups.

A new evaluation approach for uncertainty quantification methods in 6D pose estimation is proposed.

Abstract

For the use of 6D pose estimation in robotic applications, reliable poses are of utmost importance to ensure a safe, reliable and predictable operational performance. Despite these requirements, state-of-the-art 6D pose estimators often do not provide any uncertainty quantification for their pose estimates at all, or if they do, it has been shown that the uncertainty provided is only weakly correlated with the actual true error. To address this issue, we investigate a simple but effective uncertainty quantification, that we call MaskVal, which compares the pose estimates with their corresponding instance segmentations by rendering and does not require any modification of the pose estimator itself. Despite its simplicity, MaskVal significantly outperforms a state-of-the-art ensemble method on both a dataset and a robotic setup. We show that by using MaskVal, the performance of a state-of-the-art 6D pose estimator is significantly improved towards a safe and reliable operation. In addition, we propose a new and specific approach to compare and evaluate uncertainty quantification methods for 6D pose estimation in the context of robotic manipulation.