MagicStyle: Portrait Stylization Based on Reference Image
Zhaoli Deng, Kaibin Zhou, Fanyi Wang, Zhenpeng Mi

TL;DR
MagicStyle is a diffusion model-based method for portrait stylization that effectively combines content details with style textures and colors through a two-phase process involving inversion and feature fusion.
Contribution
We introduce MagicStyle, a novel diffusion model approach with a two-phase process and feature fusion attention for improved portrait stylization based on reference images.
Findings
Outperforms existing stylization methods in preserving content details.
Effectively integrates style textures and colors into portraits.
Validated through comprehensive experiments and ablation studies.
Abstract
The development of diffusion models has significantly advanced the research on image stylization, particularly in the area of stylizing a content image based on a given style image, which has attracted many scholars. The main challenge in this reference image stylization task lies in how to maintain the details of the content image while incorporating the color and texture features of the style image. This challenge becomes even more pronounced when the content image is a portrait which has complex textural details. To address this challenge, we propose a diffusion model-based reference image stylization method specifically for portraits, called MagicStyle. MagicStyle consists of two phases: Content and Style DDIM Inversion (CSDI) and Feature Fusion Forward (FFF). The CSDI phase involves a reverse denoising process, where DDIM Inversion is performed separately on the content image and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHandwritten Text Recognition Techniques · Human Motion and Animation · Image Retrieval and Classification Techniques
MethodsSoftmax · Attention Is All You Need · Fast Feedforward Networks · Diffusion
