VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance

TL;DR

This paper introduces VQGAN-CLIP, a method that generates and edits high-quality images from complex text prompts without additional training, outperforming previous models in flexibility and visual quality.

Contribution

The authors present a novel approach combining VQGAN and CLIP for open domain image generation and editing without training, enhancing flexibility and output quality.

Findings

Produces higher quality images than DALL-E, GLIDE, and Open-Edit

Operates without additional training for specific tasks

Effective for complex semantic text prompts

Abstract

Generating and editing images from open domain text prompts is a challenging task that heretofore has required expensive and specially trained models. We demonstrate a novel methodology for both tasks which is capable of producing images of high visual quality from text prompts of significant semantic complexity without any training by using a multimodal encoder to guide image generations. We demonstrate on a variety of tasks how using CLIP [37] to guide VQGAN [11] produces higher visual quality outputs than prior, less flexible approaches like DALL-E [38], GLIDE [33] and Open-Edit [24], despite not being trained for the tasks presented. Our code is available in a public repository.