Domain Generalizable Portrait Style Transfer

TL;DR

This paper introduces a novel portrait style transfer method that achieves high-quality, domain-generalized stylization with semantic alignment using a combination of semantic correspondence, AdaIN-Wavelet transform, and a dual-conditional diffusion model.

Contribution

It proposes a new framework combining semantic correspondence, AdaIN-Wavelet transform, and diffusion models for robust, high-quality portrait style transfer across diverse domains.

Findings

Outperforms existing style transfer methods in quality and domain generalization.

Effectively preserves semantic regions like hair, eyes, and lips during stylization.

Demonstrates superior results through extensive experiments.

Abstract

This paper presents a portrait style transfer method that generalizes well to various different domains while enabling high-quality semantic-aligned stylization on regions including hair, eyes, eyelashes, skins, lips, and background. To this end, we propose to establish dense semantic correspondence between the given input and reference portraits based on a pre-trained model and a semantic adapter, with which we obtain a warped reference semantically aligned with the input. To ensure effective yet controllable style transfer, we devise an AdaIN-Wavelet transform to balance content preservation and stylization by blending low-frequency information of the warped reference with high-frequency information of the input in the latent space. A style adapter is also designed to provide style guidance from the warped reference. With the stylized latent from AdaIN-Wavelet transform, we employ a dual-conditional diffusion model that integrates a ControlNet recording high-frequency information and the style guidance to generate the final result. Extensive experiments demonstrate the superiority of our method. Our code and trained model are available at https://github.com/wangxb29/DGPST.