Differential pose optimization in descriptor space -- Combining Geometric and Photometric Methods for Motion Estimation

TL;DR

This paper proposes a unified approach to two-frame pose optimization by combining photometric and geometric descriptors, aiming to improve accuracy and robustness in motion estimation.

Contribution

It introduces a method that integrates dense geometric feature descriptors with photometric residuals, blending the strengths of both paradigms in pose optimization.

Findings

The combined approach yields accurate tracking but does not outperform re-projection error methods.

Descriptor similarity metrics may be too slowly varying, affecting accuracy.

Analysis suggests limitations in the descriptor-based similarity measure.

Abstract

One of the fundamental problems in computer vision is the two-frame relative pose optimization problem. Primarily, two different kinds of error values are used: photometric error and re-projection error. The selection of error value is usually directly dependent on the selection of feature paradigm, photometric features, or geometric features. It is a trade-off between accuracy, robustness, and the possibility of loop closing. We investigate a third method that combines the strengths of both paradigms into a unified approach. Using densely sampled geometric feature descriptors, we replace the photometric error with a descriptor residual from a dense set of descriptors, thereby enabling the employment of sub-pixel accuracy in differential photometric methods, along with the expressiveness of the geometric feature descriptor. Experiments show that although the proposed strategy is an interesting approach that results in accurate tracking, it ultimately does not outperform pose optimization strategies based on re-projection error despite utilizing more information. We proceed to analyze the underlying reason for this discrepancy and present the hypothesis that the descriptor similarity metric is too slowly varying and does not necessarily correspond strictly to keypoint placement accuracy.