InstaFace: Identity-Preserving Facial Editing with Single Image Inference

TL;DR

InstaFace is a diffusion-based framework that enables realistic facial editing from a single image while effectively preserving identity and contextual features, addressing limitations of previous methods.

Contribution

We introduce a novel diffusion model with an efficient guidance network and feature embedding modules for identity-preserving facial editing from a single image.

Findings

Outperforms state-of-the-art in identity preservation

Achieves high photorealism in edited images

Effective control over pose, expression, and lighting

Abstract

Facial appearance editing is crucial for digital avatars, AR/VR, and personalized content creation, driving realistic user experiences. However, preserving identity with generative models is challenging, especially in scenarios with limited data availability. Traditional methods often require multiple images and still struggle with unnatural face shifts, inconsistent hair alignment, or excessive smoothing effects. To overcome these challenges, we introduce a novel diffusion-based framework, InstaFace, to generate realistic images while preserving identity using only a single image. Central to InstaFace, we introduce an efficient guidance network that harnesses 3D perspectives by integrating multiple 3DMM-based conditionals without introducing additional trainable parameters. Moreover, to ensure maximum identity retention as well as preservation of background, hair, and other contextual features like accessories, we introduce a novel module that utilizes feature embeddings from a facial recognition model and a pre-trained vision-language model. Quantitative evaluations demonstrate that our method outperforms several state-of-the-art approaches in terms of identity preservation, photorealism, and effective control of pose, expression, and lighting.