Condition numbers in multiview geometry, instability in relative pose estimation, and RANSAC

TL;DR

This paper develops a framework to analyze the numerical stability of minimal problems in multiview geometry, revealing why RANSAC-based relative pose estimation can fail even with clean data due to intrinsic instabilities.

Contribution

It introduces a novel approach combining algebraic and geometric tools to characterize and test the stability of minimal problems in multiview geometry, explaining RANSAC failures.

Findings

Identifies conditions leading to infinite and high condition numbers in pose estimation.

Provides computational tests to assess problem stability before solving.

Shows RANSAC favors well-conditioned data, improving robustness.

Abstract

In this paper, we introduce a general framework for analyzing the numerical conditioning of minimal problems in multiple view geometry, using tools from computational algebra and Riemannian geometry. Special motivation comes from the fact that relative pose estimation, based on standard 5-point or 7-point Random Sample Consensus (RANSAC) algorithms, can fail even when no outliers are present and there is enough data to support a hypothesis. We argue that these cases arise due to the intrinsic instability of the 5- and 7-point minimal problems. We apply our framework to characterize the instabilities, both in terms of the world scenes that lead to infinite condition number, and directly in terms of ill-conditioned image data. The approach produces computational tests for assessing the condition number before solving the minimal problem. Lastly, synthetic and real data experiments suggest that RANSAC serves not only to remove outliers, but in practice it also selects for well-conditioned image data, which is consistent with our theory.