DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models

TL;DR

DreamStyler is a new framework that enhances artistic image synthesis by combining text prompts and style references, overcoming verbal description limitations with multi-stage embedding and style transfer techniques.

Contribution

It introduces a novel multi-stage textual embedding approach and style transfer capability for improved artistic image synthesis.

Findings

Superior image quality in artistic synthesis

Effective style transfer with content and style guidance

Versatile performance across multiple artistic scenarios

Abstract

Recent progresses in large-scale text-to-image models have yielded remarkable accomplishments, finding various applications in art domain. However, expressing unique characteristics of an artwork (e.g. brushwork, colortone, or composition) with text prompts alone may encounter limitations due to the inherent constraints of verbal description. To this end, we introduce DreamStyler, a novel framework designed for artistic image synthesis, proficient in both text-to-image synthesis and style transfer. DreamStyler optimizes a multi-stage textual embedding with a context-aware text prompt, resulting in prominent image quality. In addition, with content and style guidance, DreamStyler exhibits flexibility to accommodate a range of style references. Experimental results demonstrate its superior performance across multiple scenarios, suggesting its promising potential in artistic product creation.