VAPO: Visibility-Aware Keypoint Localization for Efficient 6DoF Object Pose Estimation

TL;DR

VAPO introduces a visibility-aware approach to keypoint localization that improves 6DoF object pose estimation accuracy by effectively handling invisible keypoints, using a novel visibility importance measure integrated into existing algorithms.

Contribution

The paper proposes a method to generate visibility labels from object annotations and incorporates visibility importance into pose estimation, enhancing performance in both CAD-based and CAD-free settings.

Findings

Achieves state-of-the-art results on Linemod, Linemod-Occlusion, and YCB-V datasets.

Effectively handles invisible keypoints to improve correspondence accuracy.

Demonstrates robustness in various pose estimation scenarios.

Abstract

Localizing predefined 3D keypoints in a 2D image is an effective way to establish 3D-2D correspondences for instance-level 6DoF object pose estimation. However, unreliable localization results of invisible keypoints degrade the quality of correspondences. In this paper, we address this issue by localizing the important keypoints in terms of visibility. Since keypoint visibility information is currently missing in the dataset collection process, we propose an efficient way to generate binary visibility labels from available object-level annotations, for keypoints of both asymmetric objects and symmetric objects. We further derive real-valued visibility-aware importance from binary labels based on the PageRank algorithm. Taking advantage of the flexibility of our visibility-aware importance, we construct VAPO (Visibility-Aware POse estimator) by integrating the visibility-aware importance with a state-of-the-art pose estimation algorithm, along with additional positional encoding. VAPO can work in both CAD-based and CAD-free settings. Extensive experiments are conducted on popular pose estimation benchmarks including Linemod, Linemod-Occlusion, and YCB-V, demonstrating that VAPO clearly achieves state-of-the-art performances. Project page: https://github.com/RuyiLian/VAPO.