FastFace: Tuning Identity Preservation in Distilled Diffusion via Guidance and Attention

TL;DR

FastFace introduces a universal framework that enhances identity preservation in distilled diffusion models through guidance and attention adjustments, enabling training-free adaptation and improved generation fidelity.

Contribution

It presents a novel FastFace framework for training-free adaptation of identity-preserving adapters in distilled diffusion models, with redesigned guidance and attention mechanisms.

Findings

Improved identity similarity and fidelity in generated images.

Effective training-free adaptation of pretrained adapters.

A new evaluation protocol for identity-preserving adapters.

Abstract

In latest years plethora of identity-preserving adapters for a personalized generation with diffusion models have been released. Their main disadvantage is that they are dominantly trained jointly with base diffusion models, which suffer from slow multi-step inference. This work aims to tackle the challenge of training-free adaptation of pretrained ID-adapters to diffusion models accelerated via distillation - through careful re-design of classifier-free guidance for few-step stylistic generation and attention manipulation mechanisms in decoupled blocks to improve identity similarity and fidelity, we propose universal FastFace framework. Additionally, we develop a disentangled public evaluation protocol for id-preserving adapters.