High-resolution open-vocabulary object 6D pose estimation

TL;DR

Horyon is a novel open-vocabulary VLM-based system that accurately estimates 6D poses of unseen objects using textual prompts, achieving state-of-the-art results across multiple datasets.

Contribution

The paper introduces Horyon, a new architecture that leverages vision-language models for high-resolution, open-vocabulary 6D pose estimation of unseen objects using only textual descriptions.

Findings

Achieves state-of-the-art performance on all tested datasets.

Outperforms previous methods by 12.6 in Average Recall.

Effectively uses textual prompts for object identification and pose estimation.

Abstract

The generalisation to unseen objects in the 6D pose estimation task is very challenging. While Vision-Language Models (VLMs) enable using natural language descriptions to support 6D pose estimation of unseen objects, these solutions underperform compared to model-based methods. In this work we present Horyon, an open-vocabulary VLM-based architecture that addresses relative pose estimation between two scenes of an unseen object, described by a textual prompt only. We use the textual prompt to identify the unseen object in the scenes and then obtain high-resolution multi-scale features. These features are used to extract cross-scene matches for registration. We evaluate our model on a benchmark with a large variety of unseen objects across four datasets, namely REAL275, Toyota-Light, Linemod, and YCB-Video. Our method achieves state-of-the-art performance on all datasets, outperforming by 12.6 in Average Recall the previous best-performing approach.