Joint optimization of fitting & matching in multi-view reconstruction

TL;DR

This paper introduces a joint optimization framework for feature matching and multi-model fitting in multi-view reconstruction, improving match detection and model accuracy over traditional methods.

Contribution

It proposes fit-&-match energy formulations and an efficient min-cost-max-flow solver for joint optimization, enhancing multi-view reconstruction quality.

Findings

Increases the number of detected matches.

Allows larger view-point distances.

Improves model fitting accuracy.

Abstract

Many standard approaches for geometric model fitting are based on pre-matched image features. Typically, such pre-matching uses only feature appearances (e.g. SIFT) and a large number of non-unique features must be discarded in order to control the false positive rate. In contrast, we solve feature matching and multi-model fitting problems in a joint optimization framework. This paper proposes several fit-&-match energy formulations based on a generalization of the assignment problem. We developed an efficient solver based on min-cost-max-flow algorithm that finds near optimal solutions. Our approach significantly increases the number of detected matches. In practice, energy-based joint fitting & matching allows to increase the distance between view-points previously restricted by robustness of local SIFT-matching and to improve the model fitting accuracy when compared to state-of-the-art multi-model fitting techniques.