3D Clothed Human Reconstruction in the Wild

TL;DR

ClothWild is a novel framework for 3D clothed human reconstruction from in-the-wild images, using weak supervision and DensePose-based loss to improve robustness and accuracy over existing methods.

Contribution

The paper introduces ClothWild, a weakly supervised approach with DensePose loss to enhance 3D human reconstruction in diverse, real-world images.

Findings

Outperforms state-of-the-art methods on public in-the-wild datasets.

Demonstrates robustness to diverse poses and appearances.

Achieves more accurate reconstructions with weak supervision.

Abstract

Although much progress has been made in 3D clothed human reconstruction, most of the existing methods fail to produce robust results from in-the-wild images, which contain diverse human poses and appearances. This is mainly due to the large domain gap between training datasets and in-the-wild datasets. The training datasets are usually synthetic ones, which contain rendered images from GT 3D scans. However, such datasets contain simple human poses and less natural image appearances compared to those of real in-the-wild datasets, which makes generalization of it to in-the-wild images extremely challenging. To resolve this issue, in this work, we propose ClothWild, a 3D clothed human reconstruction framework that firstly addresses the robustness on in-thewild images. First, for the robustness to the domain gap, we propose a weakly supervised pipeline that is trainable with 2D supervision targets of in-the-wild datasets. Second, we design a DensePose-based loss function to reduce ambiguities of the weak supervision. Extensive empirical tests on several public in-the-wild datasets demonstrate that our proposed ClothWild produces much more accurate and robust results than the state-of-the-art methods. The codes are available in here: https://github.com/hygenie1228/ClothWild_RELEASE.