TetraTSDF: 3D human reconstruction from a single image with a tetrahedral outer shell

TL;DR

This paper introduces TetraTSDF, a compact tetrahedral volumetric model, and PCN, a network for detailed 3D human shape reconstruction from single images, overcoming memory and detail limitations of previous methods.

Contribution

The paper presents a novel tetrahedral TSDF model and a part connection network for accurate, detailed 3D human reconstruction from a single RGB image.

Findings

Reconstructs detailed human shapes with loose clothing from single images.

Achieves compact and dense 3D representations suitable for CNN regression.

Outperforms existing methods in detail preservation and efficiency.

Abstract

Recovering the 3D shape of a person from its 2D appearance is ill-posed due to ambiguities. Nevertheless, with the help of convolutional neural networks (CNN) and prior knowledge on the 3D human body, it is possible to overcome such ambiguities to recover detailed 3D shapes of human bodies from single images. Current solutions, however, fail to reconstruct all the details of a person wearing loose clothes. This is because of either (a) huge memory requirement that cannot be maintained even on modern GPUs or (b) the compact 3D representation that cannot encode all the details. In this paper, we propose the tetrahedral outer shell volumetric truncated signed distance function (TetraTSDF) model for the human body, and its corresponding part connection network (PCN) for 3D human body shape regression. Our proposed model is compact, dense, accurate, and yet well suited for CNN-based regression task. Our proposed PCN allows us to learn the distribution of the TSDF in the tetrahedral volume from a single image in an end-to-end manner. Results show that our proposed method allows to reconstruct detailed shapes of humans wearing loose clothes from single RGB images.