StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models

TL;DR

StyleBlend is a novel method that improves text-to-image diffusion models by learning and blending style representations from limited reference images, resulting in more coherent and style-specific image synthesis.

Contribution

It introduces a style decomposition and dual-branch synthesis approach that enhances style control and alignment in generated images compared to prior methods.

Findings

Outperforms existing methods in style coherence and content alignment

Effectively blends multiple styles through shared features

Demonstrates superior qualitative and quantitative results

Abstract

Synthesizing visually impressive images that seamlessly align both text prompts and specific artistic styles remains a significant challenge in Text-to-Image (T2I) diffusion models. This paper introduces StyleBlend, a method designed to learn and apply style representations from a limited set of reference images, enabling content synthesis of both text-aligned and stylistically coherent. Our approach uniquely decomposes style into two components, composition and texture, each learned through different strategies. We then leverage two synthesis branches, each focusing on a corresponding style component, to facilitate effective style blending through shared features without affecting content generation. StyleBlend addresses the common issues of text misalignment and weak style representation that previous methods have struggled with. Extensive qualitative and quantitative comparisons demonstrate the superiority of our approach.