Consensus Learning with Deep Sets for Essential Matrix Estimation

TL;DR

This paper introduces a simplified Deep Sets-based neural network for essential matrix estimation, outperforming complex existing methods by effectively identifying outliers and modeling noise in point matches.

Contribution

The paper presents a novel, simpler Deep Sets architecture for essential matrix estimation that improves accuracy over more complex models.

Findings

Outperforms existing deep learning methods in essential matrix estimation

Effectively identifies outliers in point matches

Models displacement noise for improved accuracy

Abstract

Robust estimation of the essential matrix, which encodes the relative position and orientation of two cameras, is a fundamental step in structure from motion pipelines. Recent deep-based methods achieved accurate estimation by using complex network architectures that involve graphs, attention layers, and hard pruning steps. Here, we propose a simpler network architecture based on Deep Sets. Given a collection of point matches extracted from two images, our method identifies outlier point matches and models the displacement noise in inlier matches. A weighted DLT module uses these predictions to regress the essential matrix. Our network achieves accurate recovery that is superior to existing networks with significantly more complex architectures.