LOHO: Latent Optimization of Hairstyles via Orthogonalization

TL;DR

LOHO introduces a novel GAN inversion-based method for hairstyle transfer that disentangles hair attributes and enables photorealistic, identity-preserving hairstyle synthesis with superior quality compared to existing methods.

Contribution

The paper presents LOHO, a new optimization-based framework that decomposes hair into attributes and uses orthogonalization for disentangled latent space manipulation.

Findings

LOHO achieves superior FID scores over SOTA hairstyle transfer methods.

LOHO preserves subject identity effectively, comparable to existing image embedding pipelines.

LOHO allows flexible manipulation of hair attributes for customized hairstyle synthesis.

Abstract

Hairstyle transfer is challenging due to hair structure differences in the source and target hair. Therefore, we propose Latent Optimization of Hairstyles via Orthogonalization (LOHO), an optimization-based approach using GAN inversion to infill missing hair structure details in latent space during hairstyle transfer. Our approach decomposes hair into three attributes: perceptual structure, appearance, and style, and includes tailored losses to model each of these attributes independently. Furthermore, we propose two-stage optimization and gradient orthogonalization to enable disentangled latent space optimization of our hair attributes. Using LOHO for latent space manipulation, users can synthesize novel photorealistic images by manipulating hair attributes either individually or jointly, transferring the desired attributes from reference hairstyles. LOHO achieves a superior FID compared with the current state-of-the-art (SOTA) for hairstyle transfer. Additionally, LOHO preserves the subject's identity comparably well according to PSNR and SSIM when compared to SOTA image embedding pipelines. Code is available at https://github.com/dukebw/LOHO.