Diff-DOPE: Differentiable Deep Object Pose Estimation

TL;DR

Diff-DOPE is a novel, training-free 6-DoF pose refinement method that uses differentiable rendering and gradient descent to achieve state-of-the-art accuracy in object pose estimation.

Contribution

It introduces a differentiable rendering-based pose refinement approach that avoids training, improving accuracy and robustness over existing neural network-based methods.

Findings

Achieves state-of-the-art results on pose estimation datasets.

Effective with multiple modalities like RGB, depth, and masks.

Avoids training by using differentiable rendering and gradient descent.

Abstract

We introduce Diff-DOPE, a 6-DoF pose refiner that takes as input an image, a 3D textured model of an object, and an initial pose of the object. The method uses differentiable rendering to update the object pose to minimize the visual error between the image and the projection of the model. We show that this simple, yet effective, idea is able to achieve state-of-the-art results on pose estimation datasets. Our approach is a departure from recent methods in which the pose refiner is a deep neural network trained on a large synthetic dataset to map inputs to refinement steps. Rather, our use of differentiable rendering allows us to avoid training altogether. Our approach performs multiple gradient descent optimizations in parallel with different random learning rates to avoid local minima from symmetric objects, similar appearances, or wrong step size. Various modalities can be used, e.g., RGB, depth, intensity edges, and object segmentation masks. We present experiments examining the effect of various choices, showing that the best results are found when the RGB image is accompanied by an object mask and depth image to guide the optimization process.