Geometry-Aware Feature Matching for Large-Scale Structure from Motion

TL;DR

This paper introduces a geometry-aware optimization approach that combines color and geometric cues to improve feature matching in large-scale Structure from Motion, especially under challenging view changes and minimal overlap.

Contribution

It presents a novel hybrid method that integrates geometric verification into feature matching, enhancing accuracy and density in large-scale SfM scenarios.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Enables effective feature matching in extreme large-scale settings.

Improves camera pose accuracy and point cloud density.

Abstract

Establishing consistent and dense correspondences across multiple images is crucial for Structure from Motion (SfM) systems. Significant view changes, such as air-to-ground with very sparse view overlap, pose an even greater challenge to the correspondence solvers. We present a novel optimization-based approach that significantly enhances existing feature matching methods by introducing geometry cues in addition to color cues. This helps fill gaps when there is less overlap in large-scale scenarios. Our method formulates geometric verification as an optimization problem, guiding feature matching within detector-free methods and using sparse correspondences from detector-based methods as anchor points. By enforcing geometric constraints via the Sampson Distance, our approach ensures that the denser correspondences from detector-free methods are geometrically consistent and more accurate. This hybrid strategy significantly improves correspondence density and accuracy, mitigates multi-view inconsistencies, and leads to notable advancements in camera pose accuracy and point cloud density. It outperforms state-of-the-art feature matching methods on benchmark datasets and enables feature matching in challenging extreme large-scale settings.