DPC-Net: Deep Pose Correction for Visual Localization

TL;DR

This paper introduces DPC-Net, a deep learning-based method that learns pose corrections to improve visual localization accuracy, effectively combining deep networks with traditional geometric algorithms.

Contribution

It proposes a novel loss function for SE(3) correction learning and demonstrates significant accuracy improvements in visual odometry using the KITTI dataset.

Findings

Enhanced visual odometry accuracy comparable to dense estimators

Effective correction of lens distortion effects

Efficient integration of deep learning with geometric localization

Abstract

We present a novel method to fuse the power of deep networks with the computational efficiency of geometric and probabilistic localization algorithms. In contrast to other methods that completely replace a classical visual estimator with a deep network, we propose an approach that uses a convolutional neural network to learn difficult-to-model corrections to the estimator from ground-truth training data. To this end, we derive a novel loss function for learning SE(3) corrections based on a matrix Lie groups approach, with a natural formulation for balancing translation and rotation errors. We use this loss to train a Deep Pose Correction network (DPC-Net) that predicts corrections for a particular estimator, sensor and environment. Using the KITTI odometry dataset, we demonstrate significant improvements to the accuracy of a computationally-efficient sparse stereo visual odometry pipeline, that render it as accurate as a modern computationally-intensive dense estimator. Further, we show how DPC-Net can be used to mitigate the effect of poorly calibrated lens distortion parameters.