TextCLIP: Text-Guided Face Image Generation And Manipulation Without Adversarial Training

TL;DR

TextCLIP is a unified, non-adversarial framework that enables high-resolution text-guided face image generation and manipulation, outperforming existing methods on multiple tasks.

Contribution

It introduces a novel framework combining StyleGAN and CLIP for text-guided image tasks without adversarial training, applicable to both generation and manipulation.

Findings

Outperforms state-of-the-art methods on CelebA-HQ dataset

Generates high-resolution images up to 1024x1024

Works effectively for both image generation and manipulation tasks

Abstract

Text-guided image generation aimed to generate desired images conditioned on given texts, while text-guided image manipulation refers to semantically edit parts of a given image based on specified texts. For these two similar tasks, the key point is to ensure image fidelity as well as semantic consistency. Many previous approaches require complex multi-stage generation and adversarial training, while struggling to provide a unified framework for both tasks. In this work, we propose TextCLIP, a unified framework for text-guided image generation and manipulation without adversarial training. The proposed method accepts input from images or random noise corresponding to these two different tasks, and under the condition of the specific texts, a carefully designed mapping network that exploits the powerful generative capabilities of StyleGAN and the text image representation capabilities of Contrastive Language-Image Pre-training (CLIP) generates images of up to $1024\times1024$ resolution that can currently be generated. Extensive experiments on the Multi-modal CelebA-HQ dataset have demonstrated that our proposed method outperforms existing state-of-the-art methods, both on text-guided generation tasks and manipulation tasks.