SHARP: Shape-Aware Reconstruction of People in Loose Clothing

TL;DR

SHARP is an innovative deep learning framework that accurately reconstructs 3D human shapes and appearances from monocular images, effectively handling loose clothing and self-occlusions with a novel fusion strategy.

Contribution

The paper introduces SHARP, a new end-to-end trainable network combining parametric and non-parametric representations for detailed 3D human reconstruction from single images.

Findings

SHARP outperforms existing methods on 3DHumans and other datasets.

The fusion strategy enables efficient training and accurate modeling of loose clothing.

The 3DHumans dataset provides rich geometrical and textural details for benchmarking.

Abstract

Recent advancements in deep learning have enabled 3D human body reconstruction from a monocular image, which has broad applications in multiple domains. In this paper, we propose SHARP (SHape Aware Reconstruction of People in loose clothing), a novel end-to-end trainable network that accurately recovers the 3D geometry and appearance of humans in loose clothing from a monocular image. SHARP uses a sparse and efficient fusion strategy to combine parametric body prior with a non-parametric 2D representation of clothed humans. The parametric body prior enforces geometrical consistency on the body shape and pose, while the non-parametric representation models loose clothing and handle self-occlusions as well. We also leverage the sparseness of the non-parametric representation for faster training of our network while using losses on 2D maps. Another key contribution is 3DHumans, our new life-like dataset of 3D human body scans with rich geometrical and textural details. We evaluate SHARP on 3DHumans and other publicly available datasets and show superior qualitative and quantitative performance than existing state-of-the-art methods.