Towards Unified 3D Hair Reconstruction from Single-View Portraits

TL;DR

This paper introduces a unified method for 3D hair reconstruction from single-view images, capable of handling both braided and un-braided styles by leveraging synthetic data and diffusion priors.

Contribution

The authors propose a novel unified pipeline that reconstructs diverse 3D hairstyles from a single image, overcoming previous limitations to specific hair types.

Findings

Achieves state-of-the-art results in complex hairstyle reconstruction.

Demonstrates good generalization to real images despite training on synthetic data.

Successfully reconstructs both braided and un-braided hairstyles using a unified approach.

Abstract

Single-view 3D hair reconstruction is challenging, due to the wide range of shape variations among diverse hairstyles. Current state-of-the-art methods are specialized in recovering un-braided 3D hairs and often take braided styles as their failure cases, because of the inherent difficulty to define priors for complex hairstyles, whether rule-based or data-based. We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. To achieve this, we first collect a large-scale synthetic multi-view hair dataset SynMvHair with diverse 3D hair in both braided and un-braided styles, and learn two diffusion priors specialized on hair. Then we optimize 3D Gaussian-based hair from the priors with two specially designed modules, i.e. view-wise and pixel-wise Gaussian refinement. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible and our method achieves the state-of-the-art performance in recovering complex hairstyles. It is worth to mention that our method shows good generalization ability to real images, although it learns hair priors from synthetic data.