VolumeDeform: Real-time Volumetric Non-rigid Reconstruction

TL;DR

VolumeDeform introduces a real-time volumetric method for non-rigid scene reconstruction using a single RGB-D sensor, capable of handling fast motion without pre-defined templates.

Contribution

It presents a novel unified volumetric framework for real-time non-rigid reconstruction that does not require prior shape templates and reduces drift through combined color and depth constraints.

Findings

Robust tracking of fast-moving scenes

Accurate non-rigid deformation estimation

Real-time performance at capture rate

Abstract

We present a novel approach for the reconstruction of dynamic geometric shapes using a single hand-held consumer-grade RGB-D sensor at real-time rates. Our method does not require a pre-defined shape template to start with and builds up the scene model from scratch during the scanning process. Geometry and motion are parameterized in a unified manner by a volumetric representation that encodes a distance field of the surface geometry as well as the non-rigid space deformation. Motion tracking is based on a set of extracted sparse color features in combination with a dense depth-based constraint formulation. This enables accurate tracking and drastically reduces drift inherent to standard model-to-depth alignment. We cast finding the optimal deformation of space as a non-linear regularized variational optimization problem by enforcing local smoothness and proximity to the input constraints. The problem is tackled in real-time at the camera's capture rate using a data-parallel flip-flop optimization strategy. Our results demonstrate robust tracking even for fast motion and scenes that lack geometric features.