Comparative Evaluation of 3D Reconstruction Methods for Object Pose Estimation

TL;DR

This paper introduces a benchmark to evaluate how the quality of 3D reconstructions from images affects object pose estimation accuracy, revealing that modern reconstructions are often sufficient and classical methods can be competitive.

Contribution

It presents a novel benchmark for assessing the impact of 3D reconstruction quality on pose estimation, highlighting the sufficiency of reconstructed models and the need for dedicated evaluation metrics.

Findings

Modern reconstruction methods often produce geometry sufficient for accurate pose estimation.

Standard 3D reconstruction metrics do not reliably predict pose estimation accuracy.

Classical, non-learning-based approaches can match or outperform modern learning-based methods in reconstruction and pose accuracy.

Abstract

Object pose estimation is essential to many industrial applications involving robotic manipulation, navigation, and augmented reality. Current generalizable object pose estimators, i.e., approaches that do not need to be trained per object, rely on accurate 3D models. Predominantly, CAD models are used, which can be hard to obtain in practice. At the same time, it is often possible to acquire images of an object. Naturally, this leads to the question whether 3D models reconstructed from images are sufficient to facilitate accurate object pose estimation. We aim to answer this question by proposing a novel benchmark for measuring the impact of 3D reconstruction quality on pose estimation accuracy. Our benchmark provides calibrated images for object reconstruction registered with the test images of the YCB-V dataset for pose evaluation under the BOP benchmark format. Detailed experiments with multiple state-of-the-art 3D reconstruction and object pose estimation approaches show that the geometry produced by modern reconstruction methods is often sufficient for accurate pose estimation. Our experiments lead to interesting observations: (1) Standard metrics for measuring 3D reconstruction quality are not necessarily indicative of pose estimation accuracy, which shows the need for dedicated benchmarks such as ours. (2) Classical, non-learning-based approaches can perform on par with modern learning-based reconstruction techniques and can even offer a better reconstruction time-pose accuracy tradeoff. (3) There is still a sizable gap between performance with reconstructed and with CAD models. To foster research on closing this gap, our benchmark is publicly available at https://github.com/VarunBurde/reconstruction_pose_benchmark}.