Deep Camera Pose Regression Using Pseudo-LiDAR

TL;DR

This paper demonstrates that using pseudo-LiDAR signals for camera pose regression significantly improves localization accuracy over traditional depth map methods, introducing a novel dual-stream neural network architecture called FusionLoc.

Contribution

The paper introduces FusionLoc, a dual-stream neural network that leverages pseudo-LiDAR for improved 6DOF camera pose estimation, outperforming existing RGB-D based methods.

Findings

FusionLoc outperforms RGB-D PoseNet by 0.33m and 4.35° on average.

Pseudo-LiDAR representations yield better localization accuracy than depth maps.

Using pseudo-LiDAR enhances large-scale camera localization systems.

Abstract

An accurate and robust large-scale localization system is an integral component for active areas of research such as autonomous vehicles and augmented reality. To this end, many learning algorithms have been proposed that predict 6DOF camera pose from RGB or RGB-D images. However, previous methods that incorporate depth typically treat the data the same way as RGB images, often adding depth maps as additional channels to RGB images and passing them through convolutional neural networks (CNNs). In this paper, we show that converting depth maps into pseudo-LiDAR signals, previously shown to be useful for 3D object detection, is a better representation for camera localization tasks by projecting point clouds that can accurately determine 6DOF camera pose. This is demonstrated by first comparing localization accuracies of a network operating exclusively on pseudo-LiDAR representations, with networks operating exclusively on depth maps. We then propose FusionLoc, a novel architecture that uses pseudo-LiDAR to regress a 6DOF camera pose. FusionLoc is a dual stream neural network, which aims to remedy common issues with typical 2D CNNs operating on RGB-D images. The results from this architecture are compared against various other state-of-the-art deep pose regression implementations using the 7 Scenes dataset. The findings are that FusionLoc performs better than a number of other camera localization methods, with a notable improvement being, on average, 0.33m and 4.35{\deg} more accurate than RGB-D PoseNet. By proving the validity of using pseudo-LiDAR signals over depth maps for localization, there are new considerations when implementing large-scale localization systems.