LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling

TL;DR

LoRD introduces a novel local 4D implicit representation for dynamic humans, enabling high-fidelity surface detail reconstruction and surpassing existing methods in 4D human modeling tasks.

Contribution

This paper presents the first local 4D implicit representation for dynamic clothed humans, combining local part-level latent codes with skeleton-guided inference for detailed surface modeling.

Findings

Outperforms state-of-the-art methods in 4D human reconstruction.

Effectively captures detailed surface deformations like clothing wrinkles.

Demonstrates robustness with sparse data and depth fusion applications.

Abstract

Recent progress in 4D implicit representation focuses on globally controlling the shape and motion with low dimensional latent vectors, which is prone to missing surface details and accumulating tracking error. While many deep local representations have shown promising results for 3D shape modeling, their 4D counterpart does not exist yet. In this paper, we fill this blank by proposing a novel Local 4D implicit Representation for Dynamic clothed human, named LoRD, which has the merits of both 4D human modeling and local representation, and enables high-fidelity reconstruction with detailed surface deformations, such as clothing wrinkles. Particularly, our key insight is to encourage the network to learn the latent codes of local part-level representation, capable of explaining the local geometry and temporal deformations. To make the inference at test-time, we first estimate the inner body skeleton motion to track local parts at each time step, and then optimize the latent codes for each part via auto-decoding based on different types of observed data. Extensive experiments demonstrate that the proposed method has strong capability for representing 4D human, and outperforms state-of-the-art methods on practical applications, including 4D reconstruction from sparse points, non-rigid depth fusion, both qualitatively and quantitatively.