Only-Style: Stylistic Consistency in Image Generation without Content Leakage

TL;DR

This paper introduces Only-Style, a novel method for image generation that maintains stylistic consistency while effectively preventing content leakage from reference images, with a new evaluation framework to measure success.

Contribution

We propose Only-Style, which localizes and mitigates content leakage during style transfer, and introduce an evaluation framework for style-consistent image generation.

Findings

Significant improvement over state-of-the-art in style consistency

Effective localization of content leakage during inference

Robust performance across diverse image instances

Abstract

Generating images in a consistent reference visual style remains a challenging computer vision task. State-of-the-art methods aiming for style-consistent generation struggle to effectively separate semantic content from stylistic elements, leading to content leakage from the image provided as a reference to the targets. To address this challenge, we propose Only-Style: a method designed to mitigate content leakage in a semantically coherent manner while preserving stylistic consistency. Only-Style works by localizing content leakage during inference, allowing the adaptive tuning of a parameter that controls the style alignment process, specifically within the image patches containing the subject in the reference image. This adaptive process best balances stylistic consistency with leakage elimination. Moreover, the localization of content leakage can function as a standalone component, given a reference-target image pair, allowing the adaptive tuning of any method-specific parameter that provides control over the impact of the stylistic reference. In addition, we propose a novel evaluation framework to quantify the success of style-consistent generations in avoiding undesired content leakage. Our approach demonstrates a significant improvement over state-of-the-art methods through extensive evaluation across diverse instances, consistently achieving robust stylistic consistency without undesired content leakage.