PViT-6D: Overclocking Vision Transformers for 6D Pose Estimation with Confidence-Level Prediction and Pose Tokens

TL;DR

PViT-6D leverages Vision Transformers for direct, end-to-end 6D pose estimation, introducing confidence-level prediction and pose tokens to improve accuracy, interpretability, and reliability over existing methods.

Contribution

The paper presents a novel transformer-based approach for 6D pose estimation that simplifies the pipeline and enhances performance with confidence prediction and scene complexity adaptation.

Findings

Outperforms state-of-the-art on Linemod-Occlusion and YCB-V datasets.

Provides a simple, end-to-end trainable framework.

Improves interpretability and inference reliability.

Abstract

In the current state of 6D pose estimation, top-performing techniques depend on complex intermediate correspondences, specialized architectures, and non-end-to-end algorithms. In contrast, our research reframes the problem as a straightforward regression task by exploring the capabilities of Vision Transformers for direct 6D pose estimation through a tailored use of classification tokens. We also introduce a simple method for determining pose confidence, which can be readily integrated into most 6D pose estimation frameworks. This involves modifying the transformer architecture by decreasing the number of query elements based on the network's assessment of the scene complexity. Our method that we call Pose Vision Transformer or PViT-6D provides the benefits of simple implementation and being end-to-end learnable while outperforming current state-of-the-art methods by +0.3% ADD(-S) on Linemod-Occlusion and +2.7% ADD(-S) on the YCB-V dataset. Moreover, our method enhances both the model's interpretability and the reliability of its performance during inference.