Skeleton Driven Non-rigid Motion Tracking and 3D Reconstruction

TL;DR

This paper introduces a skeleton-based tracking method for non-rigid human motion that improves robustness and accuracy in 3D reconstruction, especially during sudden movements, using a puppet model and a synthetic dataset.

Contribution

A novel puppet model-based tracking approach utilizing skeleton priors for better initialization and handling of articulated movements in non-rigid human performance capture.

Findings

More robust tracking during sudden articulated motions

Improved 3D reconstruction accuracy over existing methods

Introduction of a synthetic dataset with ground truth annotations

Abstract

This paper presents a method which can track and 3D reconstruct the non-rigid surface motion of human performance using a moving RGB-D camera. 3D reconstruction of marker-less human performance is a challenging problem due to the large range of articulated motions and considerable non-rigid deformations. Current approaches use local optimization for tracking. These methods need many iterations to converge and may get stuck in local minima during sudden articulated movements. We propose a puppet model-based tracking approach using skeleton prior, which provides a better initialization for tracking articulated movements. The proposed approach uses an aligned puppet model to estimate correct correspondences for human performance capture. We also contribute a synthetic dataset which provides ground truth locations for frame-by-frame geometry and skeleton joints of human subjects. Experimental results show that our approach is more robust when faced with sudden articulated motions, and provides better 3D reconstruction compared to the existing state-of-the-art approaches.