D$^3$-Human: Dynamic Disentangled Digital Human from Monocular Video

TL;DR

D$^3$-Human is a novel method that reconstructs decoupled, high-quality digital human models from monocular videos, enabling applications like clothing transfer and animation by effectively separating clothing and body details.

Contribution

It introduces a combined explicit-implicit representation approach, including a new human manifold signed distance field (hmSDF), for decoupled human reconstruction from monocular videos.

Findings

Achieves high-quality decoupled human reconstruction.

Outperforms existing methods in clothing and body separation.

Enables direct application to clothing transfer and animation.

Abstract

We introduce D$^3$-Human, a method for reconstructing Dynamic Disentangled Digital Human geometry from monocular videos. Past monocular video human reconstruction primarily focuses on reconstructing undecoupled clothed human bodies or only reconstructing clothing, making it difficult to apply directly in applications such as animation production. The challenge in reconstructing decoupled clothing and body lies in the occlusion caused by clothing over the body. To this end, the details of the visible area and the plausibility of the invisible area must be ensured during the reconstruction process. Our proposed method combines explicit and implicit representations to model the decoupled clothed human body, leveraging the robustness of explicit representations and the flexibility of implicit representations. Specifically, we reconstruct the visible region as SDF and propose a novel human manifold signed distance field (hmSDF) to segment the visible clothing and visible body, and then merge the visible and invisible body. Extensive experimental results demonstrate that, compared with existing reconstruction schemes, D$^3$-Human can achieve high-quality decoupled reconstruction of the human body wearing different clothing, and can be directly applied to clothing transfer and animation.