UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References

TL;DR

UA-Pose introduces an uncertainty-aware framework for 6D object pose estimation and online object completion from partial references, improving robustness and accuracy in scenarios with incomplete object observations.

Contribution

The paper presents a novel uncertainty-aware approach that effectively estimates 6D poses and completes objects from partial references, using uncertainty to guide confidence assessment and online completion.

Findings

Significant performance improvements over existing methods on multiple datasets.

Enhanced robustness in pose estimation with partial and incomplete object observations.

Effective online object completion guided by uncertainty estimation.

Abstract

6D object pose estimation has shown strong generalizability to novel objects. However, existing methods often require either a complete, well-reconstructed 3D model or numerous reference images that fully cover the object. Estimating 6D poses from partial references, which capture only fragments of an object's appearance and geometry, remains challenging. To address this, we propose UA-Pose, an uncertainty-aware approach for 6D object pose estimation and online object completion specifically designed for partial references. We assume access to either (1) a limited set of RGBD images with known poses or (2) a single 2D image. For the first case, we initialize a partial object 3D model based on the provided images and poses, while for the second, we use image-to-3D techniques to generate an initial object 3D model. Our method integrates uncertainty into the incomplete 3D model, distinguishing between seen and unseen regions. This uncertainty enables confidence assessment in pose estimation and guides an uncertainty-aware sampling strategy for online object completion, enhancing robustness in pose estimation accuracy and improving object completeness. We evaluate our method on the YCB-Video, YCBInEOAT, and HO3D datasets, including RGBD sequences of YCB objects manipulated by robots and human hands. Experimental results demonstrate significant performance improvements over existing methods, particularly when object observations are incomplete or partially captured. Project page: https://minfenli.github.io/UA-Pose/