NGD: Neural Gradient Based Deformation for Monocular Garment Reconstruction

TL;DR

This paper introduces NGD, a neural gradient-based deformation method for high-quality dynamic garment reconstruction from monocular videos, addressing limitations of existing implicit and explicit methods.

Contribution

The paper proposes a novel neural gradient-based deformation technique with adaptive remeshing and dynamic texture learning for improved garment reconstruction.

Findings

Significant improvements over state-of-the-art methods.

High-quality reconstruction of wrinkles and pleats.

Effective modeling of lighting and shadow effects.

Abstract

Dynamic garment reconstruction from monocular video is an important yet challenging task due to the complex dynamics and unconstrained nature of the garments. Recent advancements in neural rendering have enabled high-quality geometric reconstruction with image/video supervision. However, implicit representation methods that use volume rendering often provide smooth geometry and fail to model high-frequency details. While template reconstruction methods model explicit geometry, they use vertex displacement for deformation, which results in artifacts. Addressing these limitations, we propose NGD, a Neural Gradient-based Deformation method to reconstruct dynamically evolving textured garments from monocular videos. Additionally, we propose a novel adaptive remeshing strategy for modelling dynamically evolving surfaces like wrinkles and pleats of the skirt, leading to high-quality reconstruction. Finally, we learn dynamic texture maps to capture per-frame lighting and shadow effects. We provide extensive qualitative and quantitative evaluations to demonstrate significant improvements over existing SOTA methods and provide high-quality garment reconstructions.