Two-View Matching with View Synthesis Revisited

TL;DR

This paper revisits wide-baseline matching under extreme viewpoint changes, introducing view synthesis with affine-covariant detectors and the MODS algorithm, which adaptively balances speed and accuracy to outperform existing methods.

Contribution

It proposes the MODS algorithm that adaptively uses view synthesis and multiple detectors to improve wide-baseline matching performance and robustness.

Findings

MODS outperforms state-of-the-art methods on challenging problems.

MODS maintains comparable speed to standard matchers on simpler tasks.

View synthesis with affine detectors enhances robustness to viewpoint changes.

Abstract

Wide-baseline matching focussing on problems with extreme viewpoint change is considered. We introduce the use of view synthesis with affine-covariant detectors to solve such problems and show that matching with the Hessian-Affine or MSER detectors outperforms the state-of-the-art ASIFT. To minimise the loss of speed caused by view synthesis, we propose the Matching On Demand with view Synthesis algorithm (MODS) that uses progressively more synthesized images and more (time-consuming) detectors until reliable estimation of geometry is possible. We show experimentally that the MODS algorithm solves problems beyond the state-of-the-art and yet is comparable in speed to standard wide-baseline matchers on simpler problems. Minor contributions include an improved method for tentative correspondence selection, applicable both with and without view synthesis and a view synthesis setup greatly improving MSER robustness to blur and scale change that increase its running time by 10% only.