HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars

TL;DR

HairCUP introduces a novel 3D head avatar prior that explicitly models face and hair separately, enabling flexible, controllable, and high-fidelity avatar customization and transfer, even with limited data.

Contribution

The paper proposes a disentangled prior model for face and hair in 3D avatars, utilizing synthetic hairless data and a compositional approach for improved flexibility and generalization.

Findings

Enables seamless face and hair transfer between avatars.

Supports few-shot fine-tuning for new subjects.

Achieves high-fidelity 3D avatars with explicit hair compositionality.

Abstract

We present a universal prior model for 3D head avatars with explicit hair compositionality. Existing approaches to build generalizable priors for 3D head avatars often adopt a holistic modeling approach, treating the face and hair as an inseparable entity. This overlooks the inherent compositionality of the human head, making it difficult for the model to naturally disentangle face and hair representations, especially when the dataset is limited. Furthermore, such holistic models struggle to support applications like 3D face and hairstyle swapping in a flexible and controllable manner. To address these challenges, we introduce a prior model that explicitly accounts for the compositionality of face and hair, learning their latent spaces separately. A key enabler of this approach is our synthetic hairless data creation pipeline, which removes hair from studio-captured datasets using estimated hairless geometry and texture derived from a diffusion prior. By leveraging a paired dataset of hair and hairless captures, we train disentangled prior models for face and hair, incorporating compositionality as an inductive bias to facilitate effective separation. Our model's inherent compositionality enables seamless transfer of face and hair components between avatars while preserving identity. Additionally, we demonstrate that our model can be fine-tuned in a few-shot manner using monocular captures to create high-fidelity, hair-compositional 3D head avatars for unseen subjects. These capabilities highlight the practical applicability of our approach in real-world scenarios, paving the way for flexible and expressive 3D avatar generation.