Robust Structure from Motion in the Presence of Outliers and Missing Data

TL;DR

This paper reviews and advances techniques for non-rigid structure from motion, proposing a robust factorization method that effectively handles outliers and missing data, with experimental validation and future research directions.

Contribution

It introduces a robust affine factorization framework using homogeneous representation and outlier rejection based on reprojection residuals for non-rigid SfM.

Findings

Effective outlier rejection improves reconstruction accuracy.

The proposed method handles missing data robustly.

Experimental results validate the approach's superiority.

Abstract

Structure from motion is an import theme in computer vision. Although great progress has been made both in theory and applications, most of the algorithms only work for static scenes and rigid objects. In recent years, structure and motion recovery of non-rigid objects and dynamic scenes have received a lot of attention. In this paper, the state-of-the-art techniques for structure and motion factorization of non-rigid objects are reviewed and discussed. First, an introduction of the structure from motion problem is presented, followed by a general formulation of non-rigid structure from motion. Second, an augmented affined factorization framework, by using homogeneous representation, is presented to solve the registration issue in the presence of outlying and missing data. Third, based on the observation that the reprojection residuals of outliers are significantly larger than those of inliers, a robust factorization strategy with outlier rejection is proposed by means of the reprojection residuals, followed by some comparative experimental evaluations. Finally, some future research topics in non-rigid structure from motion are discussed.