Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

TL;DR

This paper introduces a probabilistic fusion framework combining deep neural network predictions with classical geometric solutions for relative camera pose estimation, leveraging uncertainty modeling and a graph neural network to improve accuracy.

Contribution

It presents a novel learnable fusion method guided by geometric uncertainty, integrating DNN and geometric predictions during training for enhanced pose estimation.

Findings

Achieves state-of-the-art results on DeMoN and ScanNet datasets.

Effectively models complex relationships between correspondences.

Demonstrates broad applicability for classical geometry and deep learning fusion.

Abstract

Learning methods for relative camera pose estimation have been developed largely in isolation from classical geometric approaches. The question of how to integrate predictions from deep neural networks (DNNs) and solutions from geometric solvers, such as the 5-point algorithm, has as yet remained under-explored. In this paper, we present a novel framework that involves probabilistic fusion between the two families of predictions during network training, with a view to leveraging their complementary benefits in a learnable way. The fusion is achieved by learning the DNN uncertainty under explicit guidance by the geometric uncertainty, thereby learning to take into account the geometric solution in relation to the DNN prediction. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences and potentially modeling complex relationships between points. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN and ScanNet datasets. While we focus on relative pose, we envision that our pipeline is broadly applicable for fusing classical geometry and deep learning.