DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular Videos

TL;DR

DLCA-Recon is a novel method for reconstructing dynamic human avatars with loose clothing from monocular videos, using improved initialization and deformation modeling to produce more accurate and complete 3D reconstructions.

Contribution

It introduces a dynamic deformation field with physical constraints and a better initialization strategy for accurate loose clothing reconstruction from monocular videos.

Findings

Outperforms state-of-the-art methods on public datasets

Produces more complete and realistic clothing deformations

Effectively captures free clothing movement in dynamic scenes

Abstract

Reconstructing a dynamic human with loose clothing is an important but difficult task. To address this challenge, we propose a method named DLCA-Recon to create human avatars from monocular videos. The distance from loose clothing to the underlying body rapidly changes in every frame when the human freely moves and acts. Previous methods lack effective geometric initialization and constraints for guiding the optimization of deformation to explain this dramatic change, resulting in the discontinuous and incomplete reconstruction surface. To model the deformation more accurately, we propose to initialize an estimated 3D clothed human in the canonical space, as it is easier for deformation fields to learn from the clothed human than from SMPL. With both representations of explicit mesh and implicit SDF, we utilize the physical connection information between consecutive frames and propose a dynamic deformation field (DDF) to optimize deformation fields. DDF accounts for contributive forces on loose clothing to enhance the interpretability of deformations and effectively capture the free movement of loose clothing. Moreover, we propagate SMPL skinning weights to each individual and refine pose and skinning weights during the optimization to improve skinning transformation. Based on more reasonable initialization and DDF, we can simulate real-world physics more accurately. Extensive experiments on public and our own datasets validate that our method can produce superior results for humans with loose clothing compared to the SOTA methods.