Two-View Geometry Scoring Without Correspondences

TL;DR

This paper introduces FSNet, a neural network that scores two-view geometry hypotheses without relying on traditional correspondences, improving pose estimation especially with unreliable or few matches.

Contribution

We propose FSNet, a novel deep learning-based scoring method that predicts pose errors directly, enhancing two-view geometry estimation beyond conventional inlier counting.

Findings

FSNet effectively identifies good poses with few or unreliable correspondences.

Combining FSNet with MAGSAC++ achieves state-of-the-art results.

FSNet can be integrated into existing RANSAC frameworks.

Abstract

Camera pose estimation for two-view geometry traditionally relies on RANSAC. Normally, a multitude of image correspondences leads to a pool of proposed hypotheses, which are then scored to find a winning model. The inlier count is generally regarded as a reliable indicator of "consensus". We examine this scoring heuristic, and find that it favors disappointing models under certain circumstances. As a remedy, we propose the Fundamental Scoring Network (FSNet), which infers a score for a pair of overlapping images and any proposed fundamental matrix. It does not rely on sparse correspondences, but rather embodies a two-view geometry model through an epipolar attention mechanism that predicts the pose error of the two images. FSNet can be incorporated into traditional RANSAC loops. We evaluate FSNet on fundamental and essential matrix estimation on indoor and outdoor datasets, and establish that FSNet can successfully identify good poses for pairs of images with few or unreliable correspondences. Besides, we show that naively combining FSNet with MAGSAC++ scoring approach achieves state of the art results.