StyDeco: Unsupervised Style Transfer with Distilling Priors and Semantic Decoupling

TL;DR

StyDeco introduces an unsupervised style transfer framework that learns text representations tailored for visual style transfer, effectively preserving semantic structure and fine details without human supervision.

Contribution

It proposes Prior-Guided Data Distillation and Contrastive Semantic Decoupling to improve semantic preservation and stylistic fidelity in style transfer tasks.

Findings

Outperforms existing methods in style transfer benchmarks

Enhances semantic preservation during stylization

Supports a de-stylization process for extensibility

Abstract

Diffusion models have emerged as the dominant paradigm for style transfer, but their text-driven mechanism is hindered by a core limitation: it treats textual descriptions as uniform, monolithic guidance. This limitation overlooks the semantic gap between the non-spatial nature of textual descriptions and the spatially-aware attributes of visual style, often leading to the loss of semantic structure and fine-grained details during stylization. In this paper, we propose StyDeco, an unsupervised framework that resolves this limitation by learning text representations specifically tailored for the style transfer task. Our framework first employs Prior-Guided Data Distillation (PGD), a strategy designed to distill stylistic knowledge without human supervision. It leverages a powerful frozen generative model to automatically synthesize pseudo-paired data. Subsequently, we introduce Contrastive Semantic Decoupling (CSD), a task-specific objective that adapts a text encoder using domain-specific weights. CSD performs a two-class clustering in the semantic space, encouraging source and target representations to form distinct clusters. Extensive experiments on three classic benchmarks demonstrate that our framework outperforms several existing approaches in both stylistic fidelity and structural preservation, highlighting its effectiveness in style transfer with semantic preservation. In addition, our framework supports a unique de-stylization process, further demonstrating its extensibility. Our code is vailable at https://github.com/QuanjianSong/StyDeco.