Learning Analysis-by-Synthesis for 6D Pose Estimation in RGB-D Images

TL;DR

This paper introduces a learning-based analysis-by-synthesis method for 6D object pose estimation in RGB-D images, utilizing a CNN to compare observed and rendered images, improving accuracy under occlusion and noise.

Contribution

It proposes a CNN-based approach to compare images for 6D pose estimation that generalizes across objects and backgrounds, trained with maximum likelihood.

Findings

Significant improvement over state-of-the-art methods.

Effective across diverse objects and backgrounds.

Robust to occlusion and sensor noise.

Abstract

Analysis-by-synthesis has been a successful approach for many tasks in computer vision, such as 6D pose estimation of an object in an RGB-D image which is the topic of this work. The idea is to compare the observation with the output of a forward process, such as a rendered image of the object of interest in a particular pose. Due to occlusion or complicated sensor noise, it can be difficult to perform this comparison in a meaningful way. We propose an approach that "learns to compare", while taking these difficulties into account. This is done by describing the posterior density of a particular object pose with a convolutional neural network (CNN) that compares an observed and rendered image. The network is trained with the maximum likelihood paradigm. We observe empirically that the CNN does not specialize to the geometry or appearance of specific objects, and it can be used with objects of vastly different shapes and appearances, and in different backgrounds. Compared to state-of-the-art, we demonstrate a significant improvement on two different datasets which include a total of eleven objects, cluttered background, and heavy occlusion.