NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation

TL;DR

NeuWigs introduces a two-stage neural model that captures and animates volumetric hair independently from the head, enabling realistic virtual hair synthesis and animation without direct observation, advancing virtual avatar realism.

Contribution

The paper presents a novel autoencoder-as-a-tracker for low-dimensional hair state encoding and a new hair dynamics model for stable, observation-free hair animation.

Findings

Outperforms state-of-the-art in novel view synthesis

Capable of generating realistic hair animations without direct observations

Uses multi-view segmentation and differentiable rendering for disentanglement

Abstract

The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because hair has complex geometry and appearance, as well as exhibits challenging motion. In this paper, we present a two-stage approach that models hair independently from the head to address these challenges in a data-driven manner. The first stage, state compression, learns a low-dimensional latent space of 3D hair states containing motion and appearance, via a novel autoencoder-as-a-tracker strategy. To better disentangle the hair and head in appearance learning, we employ multi-view hair segmentation masks in combination with a differentiable volumetric renderer. The second stage learns a novel hair dynamics model that performs temporal hair transfer based on the discovered latent codes. To enforce higher stability while driving our dynamics model, we employ the 3D point-cloud autoencoder from the compression stage for de-noising of the hair state. Our model outperforms the state of the art in novel view synthesis and is capable of creating novel hair animations without having to rely on hair observations as a driving signal. Project page is here https://ziyanw1.github.io/neuwigs/.