Self-Calibration Supported Robust Projective Structure-from-Motion

TL;DR

This paper introduces a unified Structure-from-Motion approach that integrates self-calibration constraints into the matching process, enhancing robustness and accuracy in multiview reconstruction.

Contribution

It proposes a novel SfM method that uses self-calibration constraints to improve matching and calibration simultaneously, employing deep learning and the Dual Image of Absolute Quadric.

Findings

Robust multiview matching achieved

Accurate camera calibration demonstrated

Self-calibration constraints improve SfM robustness

Abstract

Typical Structure-from-Motion (SfM) pipelines rely on finding correspondences across images, recovering the projective structure of the observed scene and upgrading it to a metric frame using camera self-calibration constraints. Solving each problem is mainly carried out independently from the others. For instance, camera self-calibration generally assumes correct matches and a good projective reconstruction have been obtained. In this paper, we propose a unified SfM method, in which the matching process is supported by self-calibration constraints. We use the idea that good matches should yield a valid calibration. In this process, we make use of the Dual Image of Absolute Quadric projection equations within a multiview correspondence framework, in order to obtain robust matching from a set of putative correspondences. The matching process classifies points as inliers or outliers, which is learned in an unsupervised manner using a deep neural network. Together with theoretical reasoning why the self-calibration constraints are necessary, we show experimental results demonstrating robust multiview matching and accurate camera calibration by exploiting these constraints.