Incremental Non-Rigid Structure-from-Motion with Unknown Focal Length

TL;DR

This paper introduces a fast, incremental method for non-rigid 3D reconstruction from videos that handles unknown camera focal length and improves speed and accuracy over existing techniques.

Contribution

It presents a novel approach to jointly estimate camera focal length and non-rigid shapes without shape initialization, using local rigidity and MDH heuristics, with efficient SOCP optimization.

Findings

Outperforms state-of-the-art in speed and accuracy

Handles unknown focal length in NRSfM

Efficiently incorporates many points and views

Abstract

The perspective camera and the isometric surface prior have recently gathered increased attention for Non-Rigid Structure-from-Motion (NRSfM). Despite the recent progress, several challenges remain, particularly the computational complexity and the unknown camera focal length. In this paper we present a method for incremental Non-Rigid Structure-from-Motion (NRSfM) with the perspective camera model and the isometric surface prior with unknown focal length. In the template-based case, we provide a method to estimate four parameters of the camera intrinsics. For the template-less scenario of NRSfM, we propose a method to upgrade reconstructions obtained for one focal length to another based on local rigidity and the so-called Maximum Depth Heuristics (MDH). On its basis we propose a method to simultaneously recover the focal length and the non-rigid shapes. We further solve the problem of incorporating a large number of points and adding more views in MDH-based NRSfM and efficiently solve them with Second-Order Cone Programming (SOCP). This does not require any shape initialization and produces results orders of times faster than many methods. We provide evaluations on standard sequences with ground-truth and qualitative reconstructions on challenging YouTube videos. These evaluations show that our method performs better in both speed and accuracy than the state of the art.