HairCLIP: Design Your Hair by Text and Reference Image

TL;DR

HairCLIP introduces a novel text and reference image-based hair editing framework that leverages CLIP for high-quality, disentangled hair attribute manipulation, simplifying user interaction compared to traditional sketch or mask-based methods.

Contribution

The paper presents a unified hair editing framework using CLIP for text and image conditions, enabling intuitive and precise hair attribute editing without sketches or masks.

Findings

Outperforms existing methods in manipulation accuracy

Produces highly realistic and disentangled hair edits

Preserves irrelevant attributes effectively

Abstract

Hair editing is an interesting and challenging problem in computer vision and graphics. Many existing methods require well-drawn sketches or masks as conditional inputs for editing, however these interactions are neither straightforward nor efficient. In order to free users from the tedious interaction process, this paper proposes a new hair editing interaction mode, which enables manipulating hair attributes individually or jointly based on the texts or reference images provided by users. For this purpose, we encode the image and text conditions in a shared embedding space and propose a unified hair editing framework by leveraging the powerful image text representation capability of the Contrastive Language-Image Pre-Training (CLIP) model. With the carefully designed network structures and loss functions, our framework can perform high-quality hair editing in a disentangled manner. Extensive experiments demonstrate the superiority of our approach in terms of manipulation accuracy, visual realism of editing results, and irrelevant attribute preservation. Project repo is https://github.com/wty-ustc/HairCLIP.