Diffusion Features for Zero-Shot 6DoF Object Pose Estimation

TL;DR

This paper investigates the use of Latent Diffusion Model backbones for zero-shot 6DoF object pose estimation, showing significant improvements over Vision Transformer baselines across standard datasets.

Contribution

It introduces a novel approach using LDM backbones for zero-shot pose estimation and compares it with ViT models on a common framework.

Findings

Up to 27% improvement in Average Recall over ViT baseline

Effective adaptation of LDMs for pose estimation tasks

Empirical validation on three standard datasets

Abstract

Zero-shot object pose estimation enables the retrieval of object poses from images without necessitating object-specific training. In recent approaches this is facilitated by vision foundation models (VFM), which are pre-trained models that are effectively general-purpose feature extractors. The characteristics exhibited by these VFMs vary depending on the training data, network architecture, and training paradigm. The prevailing choice in this field are self-supervised Vision Transformers (ViT). This study assesses the influence of Latent Diffusion Model (LDM) backbones on zero-shot pose estimation. In order to facilitate a comparison between the two families of models on a common ground we adopt and modify a recent approach. Therefore, a template-based multi-staged method for estimating poses in a zero-shot fashion using LDMs is presented. The efficacy of the proposed approach is empirically evaluated on three standard datasets for object-specific 6DoF pose estimation. The experiments demonstrate an Average Recall improvement of up to 27% over the ViT baseline. The source code is available at: https://github.com/BvG1993/DZOP.