Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation

TL;DR

This paper introduces a novel data-driven approach for high-quality 3D motion transfer that realistically animates apparel on stylized characters, addressing limitations of previous methods by modeling apparel deformation separately from body motion.

Contribution

The authors present a new dataset and a neural pipeline that disentangles body and apparel deformations, enabling more realistic apparel animation in motion transfer tasks.

Findings

Produces superior apparel animation quality across various clothing types.

Effectively models apparel deformation using a non-linear vertex displacement field.

Disentangles body and apparel deformations for more realistic motion transfer.

Abstract

Animating stylized characters to match a reference motion sequence is a highly demanded task in film and gaming industries. Existing methods mostly focus on rigid deformations of characters' body, neglecting local deformations on the apparel driven by physical dynamics. They deform apparel the same way as the body, leading to results with limited details and unrealistic artifacts, e.g. body-apparel penetration. In contrast, we present a novel method aiming for high-quality motion transfer with realistic apparel animation. As existing datasets lack annotations necessary for generating realistic apparel animations, we build a new dataset named MMDMC, which combines stylized characters from the MikuMikuDance community with real-world Motion Capture data. We then propose a data-driven pipeline that learns to disentangle body and apparel deformations via two neural deformation modules. For body parts, we propose a geodesic attention block to effectively incorporate semantic priors into skeletal body deformation to tackle complex body shapes for stylized characters. Since apparel motion can significantly deviate from respective body joints, we propose to model apparel deformation in a non-linear vertex displacement field conditioned on its historic states. Extensive experiments show that our method produces results with superior quality for various types of apparel. Our dataset is released in https://github.com/rongakowang/MMDMC.