Generalizable Single-view Object Pose Estimation by Two-side Generating and Matching

TL;DR

This paper introduces a novel single-view object pose estimation method that leverages a diffusion model to generate and match novel views, enabling generalization to unseen objects without extensive training or 3D models.

Contribution

The proposed approach is the first to use diffusion-based view generation and two-sided matching for generalizable pose estimation from a single RGB image.

Findings

Outperforms existing methods on synthetic and real datasets

Maintains accuracy under large viewpoint changes

Operates without 3D models or multiple views

Abstract

In this paper, we present a novel generalizable object pose estimation method to determine the object pose using only one RGB image. Unlike traditional approaches that rely on instance-level object pose estimation and necessitate extensive training data, our method offers generalization to unseen objects without extensive training, operates with a single reference image of the object, and eliminates the need for 3D object models or multiple views of the object. These characteristics are achieved by utilizing a diffusion model to generate novel-view images and conducting a two-sided matching on these generated images. Quantitative experiments demonstrate the superiority of our method over existing pose estimation techniques across both synthetic and real-world datasets. Remarkably, our approach maintains strong performance even in scenarios with significant viewpoint changes, highlighting its robustness and versatility in challenging conditions. The code will be re leased at https://github.com/scy639/Gen2SM.