Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

TL;DR

This paper introduces Direct-PoseNet, a novel absolute pose regression network that uses photometric consistency and differentiable rendering to improve localization accuracy while maintaining fast inference.

Contribution

It proposes a direct matching module with photometric supervision and adopts SO(3) rotation representation, enhancing accuracy and simplifying training compared to prior APR methods.

Findings

Achieves state-of-the-art results on 7-Scenes and LLFF datasets.

Maintains low inference time with improved accuracy.

Outperforms existing single-image APR methods.

Abstract

We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we modify the rotation representation from the classical quaternion to SO(3) in pose regression, removing the need for balancing rotation and translation loss terms. As a result, our network Direct-PoseNet achieves state-of-the-art performance among all other single-image APR methods on the 7-Scenes benchmark and the LLFF dataset.