Non-rigid Reconstruction with a Single Moving RGB-D Camera

TL;DR

This paper introduces a robust non-rigid reconstruction method using a single moving RGB-D camera, leveraging background-based camera pose estimation and a multi-scale deformation graph to improve tracking and reconstruction quality.

Contribution

It proposes a novel approach that uses rigid background information for foreground tracking and introduces a multi-scale deformation graph for enhanced non-rigid reconstruction.

Findings

More robust handling of large frame-to-frame motions.

Improved reconstruction quality over state-of-the-art methods.

Provides a new synthetic dataset for evaluation.

Abstract

We present a novel non-rigid reconstruction method using a moving RGB-D camera. Current approaches use only non-rigid part of the scene and completely ignore the rigid background. Non-rigid parts often lack sufficient geometric and photometric information for tracking large frame-to-frame motion. Our approach uses camera pose estimated from the rigid background for foreground tracking. This enables robust foreground tracking in situations where large frame-to-frame motion occurs. Moreover, we are proposing a multi-scale deformation graph which improves non-rigid tracking without compromising the quality of the reconstruction. We are also contributing a synthetic dataset which is made publically available for evaluating non-rigid reconstruction methods. The dataset provides frame-by-frame ground truth geometry of the scene, the camera trajectory, and masks for background foreground. Experimental results show that our approach is more robust in handling larger frame-to-frame motions and provides better reconstruction compared to state-of-the-art approaches.