ReMu: Reconstructing Multi-layer 3D Clothed Human from Image Layers

TL;DR

ReMu introduces a novel, single-camera setup for reconstructing multi-layer 3D clothed humans, using a unified, collision-aware neural approach that is template-free and category-agnostic, enabling diverse clothing styles.

Contribution

The paper presents a new method for multi-layer 3D human reconstruction from image layers captured by a single RGB camera, without requiring templates or category-specific models.

Findings

ReMu achieves nearly penetration-free 3D garment reconstruction.

The method performs competitively with category-specific approaches.

It effectively models diverse clothing styles in a unified framework.

Abstract

The reconstruction of multi-layer 3D garments typically requires expensive multi-view capture setups and specialized 3D editing efforts. To support the creation of life-like clothed human avatars, we introduce ReMu for reconstructing multi-layer clothed humans in a new setup, Image Layers, which captures a subject wearing different layers of clothing with a single RGB camera. To reconstruct physically plausible multi-layer 3D garments, a unified 3D representation is necessary to model these garments in a layered manner. Thus, we first reconstruct and align each garment layer in a shared coordinate system defined by the canonical body pose. Afterwards, we introduce a collision-aware optimization process to address interpenetration and further refine the garment boundaries leveraging implicit neural fields. It is worth noting that our method is template-free and category-agnostic, which enables the reconstruction of 3D garments in diverse clothing styles. Through our experiments, we show that our method reconstructs nearly penetration-free 3D clothed humans and achieves competitive performance compared to category-specific methods. Project page: https://eth-ait.github.io/ReMu/