Robust 3D Human Motion Reconstruction Via Dynamic Template Construction

TL;DR

This paper introduces a graph-based framework for reconstructing high-fidelity 3D human motion from corrupted multi-view data by creating a global template and deforming it to fill in missing information.

Contribution

It proposes a novel deformable graph approach that combines local rigidity and temporal coherence for accurate 3D human shape and motion reconstruction.

Findings

Accurate reconstruction even with severe occlusions and noise

Robust performance across various scenes and motions

Effective filling of missing geometry in low-quality data

Abstract

In multi-view human body capture systems, the recovered 3D geometry or even the acquired imagery data can be heavily corrupted due to occlusions, noise, limited field of- view, etc. Direct estimation of 3D pose, body shape or motion on these low-quality data has been traditionally challenging.In this paper, we present a graph-based non-rigid shape registration framework that can simultaneously recover 3D human body geometry and estimate pose/motion at high fidelity.Our approach first generates a global full-body template by registering all poses in the acquired motion sequence.We then construct a deformable graph by utilizing the rigid components in the global template. We directly warp the global template graph back to each motion frame in order to fill in missing geometry. Specifically, we combine local rigidity and temporal coherence constraints to maintain geometry and motion consistencies. Comprehensive experiments on various scenes show that our method is accurate and robust even in the presence of drastic motions.