Deep unsupervised 3D human body reconstruction from a sparse set of landmarks

TL;DR

This paper introduces DeepMurf, an unsupervised deep learning method that reconstructs 3D human body surfaces from sparse landmarks using autoencoders, attention models, and cascaded regression, achieving accurate results on public datasets.

Contribution

It is the first to propose an unsupervised deep learning approach for 3D human body reconstruction from sparse landmarks, combining multiple neural modules and novel loss functions.

Findings

Accurately reconstructs human bodies from sparse landmarks.

Works effectively on real-world mocap datasets.

Outperforms existing methods in unsupervised settings.

Abstract

In this paper we propose the first deep unsupervised approach in human body reconstruction to estimate body surface from a sparse set of landmarks, so called DeepMurf. We apply a denoising autoencoder to estimate missing landmarks. Then we apply an attention model to estimate body joints from landmarks. Finally, a cascading network is applied to regress parameters of a statistical generative model that reconstructs body. Our set of proposed loss functions allows us to train the network in an unsupervised way. Results on four public datasets show that our approach accurately reconstructs the human body from real world mocap data.