TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation

TL;DR

TransPoser introduces a Transformer-based neural approach for joint object shape and pose estimation from RGB-D sequences, outperforming traditional optimization methods in accuracy and efficiency.

Contribution

The paper presents DeepDDF for direct depth image prediction and TransPoser, a Transformer model, for efficient joint shape and pose estimation from sequential observations.

Findings

DeepDDF achieves high accuracy in shape representation.

TransPoser outperforms previous methods in joint estimation accuracy.

The approach is effective on both synthetic and real data.

Abstract

We propose a novel method for joint estimation of shape and pose of rigid objects from their sequentially observed RGB-D images. In sharp contrast to past approaches that rely on complex non-linear optimization, we propose to formulate it as a neural optimization that learns to efficiently estimate the shape and pose. We introduce Deep Directional Distance Function (DeepDDF), a neural network that directly outputs the depth image of an object given the camera viewpoint and viewing direction, for efficient error computation in 2D image space. We formulate the joint estimation itself as a Transformer which we refer to as TransPoser. We fully leverage the tokenization and multi-head attention to sequentially process the growing set of observations and to efficiently update the shape and pose with a learned momentum, respectively. Experimental results on synthetic and real data show that DeepDDF achieves high accuracy as a category-level object shape representation and TransPoser achieves state-of-the-art accuracy efficiently for joint shape and pose estimation.