Object Pose Transformer: Unifying Unseen Object Pose Estimation

TL;DR

This paper introduces Object Pose Transformer, a unified model that combines category-level and unseen object pose estimation, leveraging multi-view geometric reasoning and contrastive embeddings for improved accuracy in RGB and depth inputs.

Contribution

The proposed Object Pose Transformer unifies absolute and relative pose estimation paradigms within a single model, enabling versatile and accurate object pose predictions without relying on predefined taxonomies.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively estimates both absolute and relative object poses.

Operates in RGB-only and RGB-D settings with camera-agnostic design.

Abstract

Learning model-free object pose estimation for unseen instances remains a fundamental challenge in 3D vision. Existing methods typically fall into two disjoint paradigms: category-level approaches predict absolute poses in a canonical space but rely on predefined taxonomies, while relative pose methods estimate cross-view transformations but cannot recover single-view absolute pose. In this work, we propose Object Pose Transformer (\ours{}), a unified feed-forward framework that bridges these paradigms through task factorization within a single model. \ours{} jointly predicts depth, point maps, camera parameters, and normalized object coordinates (NOCS) from RGB inputs, enabling both category-level absolute SA(3) pose and unseen-object relative SE(3) pose. Our approach leverages contrastive object-centric latent embeddings for canonicalization without requiring semantic labels at inference time, and uses point maps as a camera-space representation to enable multi-view relative geometric reasoning. Through cross-frame feature interaction and shared object embeddings, our model leverages relative geometric consistency across views to improve absolute pose estimation, reducing ambiguity in single-view predictions. Furthermore, \ours{} is camera-agnostic, learning camera intrinsics on-the-fly and supporting optional depth input for metric-scale recovery, while remaining fully functional in RGB-only settings. Extensive experiments on diverse benchmarks (NOCS, HouseCat6D, Omni6DPose, Toyota-Light) demonstrate state-of-the-art performance in both absolute and relative pose estimation tasks within a single unified architecture.