DProST: Dynamic Projective Spatial Transformer Network for 6D Pose Estimation

TL;DR

DProST introduces a novel 6D pose estimation approach using a projective grid and a dynamic spatial transformer that considers projective geometry, outperforming existing mesh-based methods especially in mesh-less scenarios.

Contribution

The paper proposes DProST, a mesh-less 6D pose estimation method utilizing a projective grid and a dynamic spatial transformer, addressing limitations of vertex-based objectives.

Findings

Outperforms state-of-the-art mesh-based methods on LINEMOD datasets.

Shows competitive results on YCBV dataset without using object meshes.

Effective in mesh-less settings with reconstructed features.

Abstract

Predicting the object's 6D pose from a single RGB image is a fundamental computer vision task. Generally, the distance between transformed object vertices is employed as an objective function for pose estimation methods. However, projective geometry in the camera space is not considered in those methods and causes performance degradation. In this regard, we propose a new pose estimation system based on a projective grid instead of object vertices. Our pose estimation method, dynamic projective spatial transformer network (DProST), localizes the region of interest grid on the rays in camera space and transforms the grid to object space by estimated pose. The transformed grid is used as both a sampling grid and a new criterion of the estimated pose. Additionally, because DProST does not require object vertices, our method can be used in a mesh-less setting by replacing the mesh with a reconstructed feature. Experimental results show that mesh-less DProST outperforms the state-of-the-art mesh-based methods on the LINEMOD and LINEMOD-OCCLUSION dataset, and shows competitive performance on the YCBV dataset with mesh data. The source code is available at https://github.com/parkjaewoo0611/DProST