ArtCrafter: Text-Image Aligning Style Transfer via Embedding Reframing
Nisha Huang, Kaer Huang, Yifan Pu, Jiangshan Wang, Jie Guo, Yiqiang, Yan, Xiu Li, Tong-Yee Lee

TL;DR
ArtCrafter introduces a novel text-to-image style transfer framework that uses attention-based style extraction and embedding reframing to improve stylistic intensity, controllability, and diversity in generated images.
Contribution
The paper presents a new framework combining attention-based style extraction and embedding reframing for improved text-image style transfer.
Findings
Achieves high stylistic intensity and diversity in stylized images.
Enhances controllability over style transfer results.
Demonstrates superior performance through extensive experiments.
Abstract
Recent years have witnessed significant advancements in text-guided style transfer, primarily attributed to innovations in diffusion models. These models excel in conditional guidance, utilizing text or images to direct the sampling process. However, despite their capabilities, direct conditional guidance approaches often face challenges in balancing the expressiveness of textual semantics with the diversity of output results while capturing stylistic features. To address these challenges, we introduce ArtCrafter, a novel framework for text-to-image style transfer. Specifically, we introduce an attention-based style extraction module, meticulously engineered to capture the subtle stylistic elements within an image. This module features a multi-layer architecture that leverages the capabilities of perceiver attention mechanisms to integrate fine-grained information. Additionally, we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Handwritten Text Recognition Techniques · Image Processing and 3D Reconstruction
MethodsSoftmax · Attention Is All You Need · Diffusion
