Stable-Hair: Real-World Hair Transfer via Diffusion Model

TL;DR

Stable-Hair introduces a diffusion-based two-stage framework for realistic and detailed hair transfer, effectively handling diverse hairstyles and ensuring high fidelity and identity preservation in virtual try-on applications.

Contribution

The paper proposes a novel two-stage diffusion-based framework with a Bald Converter, Hair Extractor, and Latent IdentityNet, including a Latent ControlNet architecture for high-fidelity hair transfer.

Findings

Achieves state-of-the-art hair transfer quality

Handles diverse and intricate hairstyles effectively

Maintains high fidelity and identity consistency

Abstract

Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. In this paper, we propose a novel diffusion-based hair transfer framework, named \textit{Stable-Hair}, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on. To achieve this goal, our Stable-Hair framework is designed as a two-stage pipeline. In the first stage, we train a Bald Converter alongside stable diffusion to remove hair from the user-provided face images, resulting in bald images. In the second stage, we specifically designed a Hair Extractor and a Latent IdentityNet to transfer the target hairstyle with highly detailed and high-fidelity to the bald image. The Hair Extractor is trained to encode reference images with the desired hairstyles, while the Latent IdentityNet ensures consistency in identity and background. To minimize color deviations between source images and transfer results, we introduce a novel Latent ControlNet architecture, which functions as both the Bald Converter and Latent IdentityNet. After training on our curated triplet dataset, our method accurately transfers highly detailed and high-fidelity hairstyles to the source images. Extensive experiments demonstrate that our approach achieves state-of-the-art performance compared to existing hair transfer methods. Project page: \textcolor{red}{\url{https://xiaojiu-z.github.io/Stable-Hair.github.io/}}