Refining 6D Object Pose Predictions using Abstract Render-and-Compare

TL;DR

This paper introduces a novel method that refines 6D object pose predictions in cluttered scenes by using an abstract feature space for inverse rendering, improving accuracy over existing deep-learning approaches.

Contribution

It proposes mapping rasterized images into an abstract feature space to facilitate pose refinement through inverse rendering, addressing occlusion and inter-object effects.

Findings

Significant improvements on the YCB-Video dataset.

Large basin of attraction towards correct poses.

Effective refinement in highly cluttered scenes.

Abstract

Robotic systems often require precise scene analysis capabilities, especially in unstructured, cluttered situations, as occurring in human-made environments. While current deep-learning based methods yield good estimates of object poses, they often struggle with large amounts of occlusion and do not take inter-object effects into account. Vision as inverse graphics is a promising concept for detailed scene analysis. A key element for this idea is a method for inferring scene parameter updates from the rasterized 2D scene. However, the rasterization process is notoriously difficult to invert, both due to the projection and occlusion process, but also due to secondary effects such as lighting or reflections. We propose to remove the latter from the process by mapping the rasterized image into an abstract feature space learned in a self-supervised way from pixel correspondences. Using only a light-weight inverse rendering module, this allows us to refine 6D object pose estimations in highly cluttered scenes by optimizing a simple pixel-wise difference in the abstract image representation. We evaluate our approach on the challenging YCB-Video dataset, where it yields large improvements and demonstrates a large basin of attraction towards the correct object poses.