Dense-Face: Personalized Face Generation Model via Dense Annotation Prediction

TL;DR

Dense-Face is a novel text-to-image personalization diffusion model that generates high-quality, identity-preserving face images aligned with text captions, without test-time fine-tuning.

Contribution

It introduces a pose-controllable adapter and uses internal features of the diffusion model to improve face generation and text alignment.

Findings

Achieves state-of-the-art performance in image-text alignment.

Maintains consistent identity in generated faces.

Provides effective pose control in face synthesis.

Abstract

The text-to-image (T2I) personalization diffusion model can generate images of the novel concept based on the user input text caption. However, existing T2I personalized methods either require test-time fine-tuning or fail to generate images that align well with the given text caption. In this work, we propose a new T2I personalization diffusion model, Dense-Face, which can generate face images with a consistent identity as the given reference subject and align well with the text caption. Specifically, we introduce a pose-controllable adapter for the high-fidelity image generation while maintaining the text-based editing ability of the pre-trained stable diffusion (SD). Additionally, we use internal features of the SD UNet to predict dense face annotations, enabling the proposed method to gain domain knowledge in face generation. Empirically, our method achieves state-of-the-art or competitive generation performance in image-text alignment, identity preservation, and pose control.