6-DoF Object Pose from Semantic Keypoints

TL;DR

This paper introduces a new method for estimating 6-DoF object pose from a single RGB image using semantic keypoints and a deformable shape model, effective for textured and textureless objects.

Contribution

The approach combines semantic keypoints with a deformable shape model, enabling accurate pose estimation for both instance- and class-based objects regardless of texture.

Findings

Achieves state-of-the-art accuracy on PASCAL3D+ dataset.

Works effectively with cluttered backgrounds.

Applicable to textured and textureless objects.

Abstract

This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.