FaceSnap: Enhanced ID-fidelity Network for Tuning-free Portrait Customization

TL;DR

FaceSnap is a novel portrait customization method based on Stable Diffusion that requires only one reference image, ensuring high fidelity and identity preservation without fine-tuning, and is easily extendable to different models.

Contribution

We introduce FaceSnap, a plug-and-play, tuning-free portrait generation framework with a Facial Attribute Mixer, Landmark Predictor, and ID-preserving module for superior personalized image synthesis.

Findings

Outperforms state-of-the-art methods in personalized portrait generation.

Requires only a single reference image for high-fidelity results.

Maintains identity and facial details across diverse poses.

Abstract

Benefiting from the significant advancements in text-to-image diffusion models, research in personalized image generation, particularly customized portrait generation, has also made great strides recently. However, existing methods either require time-consuming fine-tuning and lack generalizability or fail to achieve high fidelity in facial details. To address these issues, we propose FaceSnap, a novel method based on Stable Diffusion (SD) that requires only a single reference image and produces extremely consistent results in a single inference stage. This method is plug-and-play and can be easily extended to different SD models. Specifically, we design a new Facial Attribute Mixer that can extract comprehensive fused information from both low-level specific features and high-level abstract features, providing better guidance for image generation. We also introduce a Landmark Predictor that maintains reference identity across landmarks with different poses, providing diverse yet detailed spatial control conditions for image generation. Then we use an ID-preserving module to inject these into the UNet. Experimental results demonstrate that our approach performs remarkably in personalized and customized portrait generation, surpassing other state-of-the-art methods in this domain.