PBDyG: Position Based Dynamic Gaussians for Motion-Aware Clothed Human Avatars

TL;DR

This paper presents PBDyG, a novel model for clothed human avatars that uses position-based dynamics and physical simulation to accurately reproduce body and cloth movements from multiview RGB videos.

Contribution

Introducing Position Based Dynamic Gaussians (PBDyG), a new method that models clothing as physical entities driven by body movement, enabling realistic cloth deformation in avatars.

Findings

Accurately reproduces appearance and cloth deformation.

Reconstructs avatars with highly deformable garments.

Outperforms existing methods in cloth realism.

Abstract

This paper introduces a novel clothed human model that can be learned from multiview RGB videos, with a particular emphasis on recovering physically accurate body and cloth movements. Our method, Position Based Dynamic Gaussians (PBDyG), realizes ``movement-dependent'' cloth deformation via physical simulation, rather than merely relying on ``pose-dependent'' rigid transformations. We model the clothed human holistically but with two distinct physical entities in contact: clothing modeled as 3D Gaussians, which are attached to a skinned SMPL body that follows the movement of the person in the input videos. The articulation of the SMPL body also drives physically-based simulation of the clothes' Gaussians to transform the avatar to novel poses. In order to run position based dynamics simulation, physical properties including mass and material stiffness are estimated from the RGB videos through Dynamic 3D Gaussian Splatting. Experiments demonstrate that our method not only accurately reproduces appearance but also enables the reconstruction of avatars wearing highly deformable garments, such as skirts or coats, which have been challenging to reconstruct using existing methods.