StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis

TL;DR

StyleCLIPDraw enhances text-to-drawing synthesis by integrating style control, enabling artistic customization of generated images in both texture and shape through a coupled style transfer approach.

Contribution

It introduces a novel coupled style loss to the CLIPDraw model, allowing simultaneous control of content and style in generated drawings.

Findings

Coupled style transfer captures style in texture and shape.

The method improves artistic control over generated images.

Code and more results are publicly available.

Abstract

Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated. We introduce StyleCLIPDraw which adds a style loss to the CLIPDraw text-to-drawing synthesis model to allow artistic control of the synthesized drawings in addition to control of the content via text. Whereas performing decoupled style transfer on a generated image only affects the texture, our proposed coupled approach is able to capture a style in both texture and shape, suggesting that the style of the drawing is coupled with the drawing process itself. More results and our code are available at https://github.com/pschaldenbrand/StyleCLIPDraw