Dynamic Neural Garments

TL;DR

This paper introduces a neural network-based method for generating realistic dynamic garment images on digital avatars from body joint motions, improving realism and generalization over previous static or tight garment-focused models.

Contribution

It presents a novel dynamic neural garment model that directly synthesizes realistic garment sequences from motion inputs, handling unseen views and shapes with fine-tuning capabilities.

Findings

Generates plausible garment dynamics from unseen motions and views.

Outperforms existing neural rendering and image translation methods.

Can be fine-tuned for new body shapes and backgrounds.

Abstract

A vital task of the wider digital human effort is the creation of realistic garments on digital avatars, both in the form of characteristic fold patterns and wrinkles in static frames as well as richness of garment dynamics under avatars' motion. Existing workflow of modeling, simulation, and rendering closely replicates the physics behind real garments, but is tedious and requires repeating most of the workflow under changes to characters' motion, camera angle, or garment resizing. Although data-driven solutions exist, they either focus on static scenarios or only handle dynamics of tight garments. We present a solution that, at test time, takes in body joint motion to directly produce realistic dynamic garment image sequences. Specifically, given the target joint motion sequence of an avatar, we propose dynamic neural garments to jointly simulate and render plausible dynamic garment appearance from an unseen viewpoint. Technically, our solution generates a coarse garment proxy sequence, learns deep dynamic features attached to this template, and neurally renders the features to produce appearance changes such as folds, wrinkles, and silhouettes. We demonstrate generalization behavior to both unseen motion and unseen camera views. Further, our network can be fine-tuned to adopt to new body shape and/or background images. We also provide comparisons against existing neural rendering and image sequence translation approaches, and report clear quantitative improvements.