InstantID: Zero-shot Identity-Preserving Generation in Seconds

TL;DR

InstantID introduces a fast, high-fidelity, zero-shot image personalization method using a diffusion model that requires only a single facial image and integrates seamlessly with existing pre-trained models.

Contribution

We propose InstantID, a novel diffusion-based plug-and-play module that enables identity-preserving image generation from a single image without extensive fine-tuning.

Findings

Achieves high face fidelity with minimal reference images.

Compatible with popular pre-trained diffusion models like SD1.5 and SDXL.

Offers efficient, real-time image personalization in practical applications.

Abstract

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.