Do We Really Need Scene-specific Pose Encoders?

TL;DR

This paper demonstrates that scene-specific pose encoders are unnecessary for visual pose regression, showing that generic image similarity encodings can achieve competitive or superior localization results, including consistent outdoor localization within tight error bounds.

Contribution

It introduces a scene-agnostic approach using pre-computed image similarity encodings for pose regression, challenging the necessity of scene-specific encoders.

Findings

Encodings from image retrieval models suffice for accurate pose regression.

The proposed method can outperform state-of-the-art scene-specific models.

Achieves outdoor localization within 2 meters and 5 degrees consistently.

Abstract

Visual pose regression models estimate the camera pose from a query image with a single forward pass. Current models learn pose encoding from an image using deep convolutional networks which are trained per scene. The resulting encoding is typically passed to a multi-layer perceptron in order to regress the pose. In this work, we propose that scene-specific pose encoders are not required for pose regression and that encodings trained for visual similarity can be used instead. In order to test our hypothesis, we take a shallow architecture of several fully connected layers and train it with pre-computed encodings from a generic image retrieval model. We find that these encodings are not only sufficient to regress the camera pose, but that, when provided to a branching fully connected architecture, a trained model can achieve competitive results and even surpass current \textit{state-of-the-art} pose regressors in some cases. Moreover, we show that for outdoor localization, the proposed architecture is the only pose regressor, to date, consistently localizing in under 2 meters and 5 degrees.