OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models

TL;DR

This paper introduces a keypoint-free, one-shot object pose estimation method that does not require CAD models or object-specific training, outperforming previous methods especially on low-textured objects.

Contribution

The authors propose a novel keypoint-free pipeline built on LoFTR for 3D reconstruction and direct 2D-3D matching, eliminating the need for keypoint detection in pose estimation.

Findings

Outperforms existing CAD-model-free methods significantly.

Achieves comparable accuracy to CAD-model-based methods on LINEMOD.

Effective on low-textured objects.

Abstract

We propose a new method for object pose estimation without CAD models. The previous feature-matching-based method OnePose has shown promising results under a one-shot setting which eliminates the need for CAD models or object-specific training. However, OnePose relies on detecting repeatable image keypoints and is thus prone to failure on low-textured objects. We propose a keypoint-free pose estimation pipeline to remove the need for repeatable keypoint detection. Built upon the detector-free feature matching method LoFTR, we devise a new keypoint-free SfM method to reconstruct a semi-dense point-cloud model for the object. Given a query image for object pose estimation, a 2D-3D matching network directly establishes 2D-3D correspondences between the query image and the reconstructed point-cloud model without first detecting keypoints in the image. Experiments show that the proposed pipeline outperforms existing one-shot CAD-model-free methods by a large margin and is comparable to CAD-model-based methods on LINEMOD even for low-textured objects. We also collect a new dataset composed of 80 sequences of 40 low-textured objects to facilitate future research on one-shot object pose estimation. The supplementary material, code and dataset are available on the project page: https://zju3dv.github.io/onepose_plus_plus/.