ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion

TL;DR

This paper introduces ENFT, a framework for robust feature tracking across disjointed image sequences, improving large-scale structure-from-motion by handling interrupted tracks and occlusions effectively.

Contribution

The paper presents a novel non-consecutive feature tracking method and a segment-based SfM algorithm for large datasets, enhancing robustness and efficiency.

Findings

Effective handling of interrupted feature tracks in large-scale scenes

Improved robustness in SfM with challenging data

Demonstrated success on challenging video datasets

Abstract

Structure-from-motion (SfM) largely relies on feature tracking. In image sequences, if disjointed tracks caused by objects moving in and out of the field of view, occasional occlusion, or image noise, are not handled well, corresponding SfM could be affected. This problem becomes severer for large-scale scenes, which typically requires to capture multiple sequences to cover the whole scene. In this paper, we propose an efficient non-consecutive feature tracking (ENFT) framework to match interrupted tracks distributed in different subsequences or even in different videos. Our framework consists of steps of solving the feature `dropout' problem when indistinctive structures, noise or large image distortion exists, and of rapidly recognizing and joining common features located in different subsequences. In addition, we contribute an effective segment-based coarse-to-fine SfM algorithm for robustly handling large datasets. Experimental results on challenging video data demonstrate the effectiveness of the proposed system.