Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image

TL;DR

DynaAvatar is a zero-shot framework that reconstructs animatable 3D human avatars with cloth dynamics from a single image, leveraging large-scale datasets, a Transformer architecture, and a novel optical flow-guided loss.

Contribution

It introduces a static-to-dynamic knowledge transfer strategy and a new DynaFlow loss for improved cloth dynamic modeling without subject-specific optimization.

Findings

Outperforms prior methods in visual quality and animation realism

Effectively models cloth dynamics from a single image

Generalizes well across diverse subjects and motions

Abstract

Existing single-image 3D human avatar methods primarily rely on rigid joint transformations, limiting their ability to model realistic cloth dynamics. We present DynaAvatar, a zero-shot framework that reconstructs animatable 3D human avatars with motion-dependent cloth dynamics from a single image. Trained on large-scale multi-person motion datasets, DynaAvatar employs a Transformer-based feed-forward architecture that directly predicts dynamic 3D Gaussian deformations without subject-specific optimization. To overcome the scarcity of dynamic captures, we introduce a static-to-dynamic knowledge transfer strategy: a Transformer pretrained on large-scale static captures provides strong geometric and appearance priors, which are efficiently adapted to motion-dependent deformations through lightweight LoRA fine-tuning on dynamic captures. We further propose the DynaFlow loss, an optical flow-guided objective that provides reliable motion-direction geometric cues for cloth dynamics in rendered space. Finally, we reannotate the missing or noisy SMPL-X fittings in existing dynamic capture datasets, as most public dynamic capture datasets contain incomplete or unreliable fittings that are unsuitable for training high-quality 3D avatar reconstruction models. Experiments demonstrate that DynaAvatar produces visually rich and generalizable animations, outperforming prior methods.