HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion

TL;DR

HairWeaver introduces a diffusion-based method for animating a single human image with realistic, expressive hair motion, overcoming previous limitations in hair control and realism through specialized modules and a CG-simulated dataset.

Contribution

The paper proposes a novel diffusion pipeline with two lightweight modules for controlling hair motion and appearance, enabling photorealistic hair animation from a single image.

Findings

Achieves state-of-the-art realism in hair animation

Provides fine control over hair dynamics

Successfully generalizes across different data domains

Abstract

We present HairWeaver, a diffusion-based pipeline that animates a single human image with realistic and expressive hair dynamics. While existing methods successfully control body pose, they lack specific control over hair, and as a result, fail to capture the intricate hair motions, resulting in stiff and unrealistic animations. HairWeaver overcomes this limitation using two specialized modules: a Motion-Context-LoRA to integrate motion conditions and a Sim2Real-Domain-LoRA to preserve the subject's photoreal appearance across different data domains. These lightweight components are designed to guide a video diffusion backbone while maintaining its core generative capabilities. By training on a specialized dataset of dynamic human motion generated from a CG simulator, HairWeaver affords fine control over hair motion and ultimately learns to produce highly realistic hair that responds naturally to movement. Comprehensive evaluations demonstrate that our approach sets a new state of the art, producing lifelike human hair animations with dynamic details.