Expanding the Content-Style Frontier: a Balanced Subspace Blending Approach for Content-Style LoRA Fusion

TL;DR

This paper introduces a new method for text-to-image diffusion models that enhances the balance between content preservation and style application, broadening the content-style trade-off spectrum.

Contribution

It proposes Content-Style Subspace Blending and a Balance loss to improve content similarity across different style intensities, expanding the content-style frontier.

Findings

Outperforms existing methods in qualitative evaluations

Achieves lower IGD and GD scores

Provides a better content-style trade-off

Abstract

Recent advancements in text-to-image diffusion models have significantly improved the personalization and stylization of generated images. However, previous studies have only assessed content similarity under a single style intensity. In our experiments, we observe that increasing style intensity leads to a significant loss of content features, resulting in a suboptimal content-style frontier. To address this, we propose a novel approach to expand the content-style frontier by leveraging Content-Style Subspace Blending and a Content-Style Balance loss. Our method improves content similarity across varying style intensities, significantly broadening the content-style frontier. Extensive experiments demonstrate that our approach outperforms existing techniques in both qualitative and quantitative evaluations, achieving superior content-style trade-off with significantly lower Inverted Generational Distance (IGD) and Generational Distance (GD) scores compared to current methods.