CreativeSynth: Cross-Art-Attention for Artistic Image Synthesis with Multimodal Diffusion
Nisha Huang, Weiming Dong, Yuxin Zhang, Fan Tang, Ronghui Li, Chongyang Ma, Xiu Li, Tong-Yee Lee, Changsheng Xu

TL;DR
CreativeSynth introduces a novel diffusion-based framework that uses cross-art-attention to integrate multimodal semantic information into artworks, enhancing artistic synthesis while maintaining aesthetic harmony.
Contribution
It presents a new multi-task unified diffusion model that effectively combines multimodal features with attention mechanisms for artistic image synthesis.
Findings
Successfully integrates real-world semantics into artworks
Maintains aesthetic harmony during semantic editing
Works across diverse art categories
Abstract
Although remarkable progress has been made in image style transfer, style is just one of the components of artistic paintings. Directly transferring extracted style features to natural images often results in outputs with obvious synthetic traces. This is because key painting attributes including layout, perspective, shape, and semantics often cannot be conveyed and expressed through style transfer. Large-scale pretrained text-to-image generation models have demonstrated their capability to synthesize a vast amount of high-quality images. However, even with extensive textual descriptions, it is challenging to fully express the unique visual properties and details of paintings. Moreover, generic models often disrupt the overall artistic effect when modifying specific areas, making it more complicated to achieve a unified aesthetic in artworks. Our main novel idea is to integrate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Image Retrieval and Classification Techniques
MethodsDiffusion
