Averaging Essential and Fundamental Matrices in Collinear Camera Settings

TL;DR

This paper addresses the challenge of averaging essential and fundamental matrices in collinear camera configurations within Structure from Motion, proposing new algorithms that improve robustness and accuracy in such scenarios.

Contribution

It introduces spectral analysis and two novel algorithms for averaging bifocal tensors in collinear camera setups, enhancing existing methods.

Findings

Achieves state-of-the-art results on autonomous car datasets.

Effective in both calibrated and uncalibrated settings.

Handles mixed collinear and non-collinear camera configurations.

Abstract

Global methods to Structure from Motion have gained popularity in recent years. A significant drawback of global methods is their sensitivity to collinear camera settings. In this paper, we introduce an analysis and algorithms for averaging bifocal tensors (essential or fundamental matrices) when either subsets or all of the camera centers are collinear. We provide a complete spectral characterization of bifocal tensors in collinear scenarios and further propose two averaging algorithms. The first algorithm uses rank constrained minimization to recover camera matrices in fully collinear settings. The second algorithm enriches the set of possibly mixed collinear and non-collinear cameras with additional, "virtual cameras," which are placed in general position, enabling the application of existing averaging methods to the enriched set of bifocal tensors. Our algorithms are shown to achieve state of the art results on various benchmarks that include autonomous car datasets and unordered image collections in both calibrated and unclibrated settings.