Insights on Evaluation of Camera Re-localization Using Relative Pose Regression

TL;DR

This paper analyzes the evaluation of camera re-localization methods based on relative pose regression, revealing a tradeoff between accuracy and subspace volume, and proposes new metrics and a baseline network to improve assessment robustness.

Contribution

It introduces three new evaluation metrics that account for regression subspace volume and presents a new baseline network for relative pose regression.

Findings

Proposed metrics are more robust to overlap threshold variations.

The new network baseline performs well across multiple datasets.

Training on a single scene generalizes effectively to others.

Abstract

We consider the problem of relative pose regression in visual relocalization. Recently, several promising approaches have emerged in this area. We claim that even though they demonstrate on the same datasets using the same split to train and test, a faithful comparison between them was not available since on currently used evaluation metric, some approaches might perform favorably, while in reality performing worse. We reveal a tradeoff between accuracy and the 3D volume of the regressed subspace. We believe that unlike other relocalization approaches, in the case of relative pose regression, the regressed subspace 3D volume is less dependent on the scene and more affect by the method used to score the overlap, which determined how closely sampled viewpoints are. We propose three new metrics to remedy the issue mentioned above. The proposed metrics incorporate statistics about the regression subspace volume. We also propose a new pose regression network that serves as a new baseline for this task. We compare the performance of our trained model on Microsoft 7-Scenes and Cambridge Landmarks datasets both with the standard metrics and the newly proposed metrics and adjust the overlap score to reveal the tradeoff between the subspace and performance. The results show that the proposed metrics are more robust to different overlap threshold than the conventional approaches. Finally, we show that our network generalizes well, specifically, training on a single scene leads to little loss of performance on the other scenes.