MODIFY: Model-driven Face Stylization without Style Images

TL;DR

MODIFY is a novel face stylization method that does not require target style images, using a generative model and remapping network to achieve privacy-preserving, multimodal stylization.

Contribution

It introduces a model-driven approach that bypasses the need for target images and preserves style diversity, enhancing privacy and applicability.

Findings

Effective unsupervised face stylization demonstrated on multiple datasets.

Preserves multimodal style information during translation.

No reliance on target style images during inference.

Abstract

Existing face stylization methods always acquire the presence of the target (style) domain during the translation process, which violates privacy regulations and limits their applicability in real-world systems. To address this issue, we propose a new method called MODel-drIven Face stYlization (MODIFY), which relies on the generative model to bypass the dependence of the target images. Briefly, MODIFY first trains a generative model in the target domain and then translates a source input to the target domain via the provided style model. To preserve the multimodal style information, MODIFY further introduces an additional remapping network, mapping a known continuous distribution into the encoder's embedding space. During translation in the source domain, MODIFY fine-tunes the encoder module within the target style-persevering model to capture the content of the source input as precisely as possible. Our method is extremely simple and satisfies versatile training modes for face stylization. Experimental results on several different datasets validate the effectiveness of MODIFY for unsupervised face stylization.