Differentiable Rendering for Pose Estimation in Proximity Operations

TL;DR

This paper introduces a differentiable rendering-based method for 6-DoF pose estimation that compares images in feature space and learns local gradients online, enabling precise pose alignment in proximity operations.

Contribution

It proposes a novel gradient-based pose estimation algorithm that uses feature-space comparison and online learned gradients within a differentiable rendering pipeline.

Findings

Effective in proximity operation scenarios

Outperforms traditional reprojection error methods

Enables direct pose parameter regression

Abstract

Differentiable rendering aims to compute the derivative of the image rendering function with respect to the rendering parameters. This paper presents a novel algorithm for 6-DoF pose estimation through gradient-based optimization using a differentiable rendering pipeline. We emphasize two key contributions: (1) instead of solving the conventional 2D to 3D correspondence problem and computing reprojection errors, images (rendered using the 3D model) are compared only in the 2D feature space via sparse 2D feature correspondences. (2) Instead of an analytical image formation model, we compute an approximate local gradient of the rendering process through online learning. The learning data consists of image features extracted from multi-viewpoint renders at small perturbations in the pose neighborhood. The gradients are propagated through the rendering pipeline for the 6-DoF pose estimation using nonlinear least squares. This gradient-based optimization regresses directly upon the pose parameters by aligning the 3D model to reproduce a reference image shape. Using representative experiments, we demonstrate the application of our approach to pose estimation in proximity operations.