AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation

TL;DR

AttnDreamBooth is a new method that improves personalized text-to-image generation by better aligning concept embeddings, attention maps, and identity preservation, addressing limitations of existing techniques like Textual Inversion and DreamBooth.

Contribution

It introduces a multi-stage training approach with cross-attention regularization to enhance concept embedding alignment and image personalization.

Findings

Improves identity preservation in generated images.

Enhances text alignment with user prompts.

Outperforms baseline methods in personalization quality.

Abstract

Recent advances in text-to-image models have enabled high-quality personalized image synthesis of user-provided concepts with flexible textual control. In this work, we analyze the limitations of two primary techniques in text-to-image personalization: Textual Inversion and DreamBooth. When integrating the learned concept into new prompts, Textual Inversion tends to overfit the concept, while DreamBooth often overlooks it. We attribute these issues to the incorrect learning of the embedding alignment for the concept. We introduce AttnDreamBooth, a novel approach that addresses these issues by separately learning the embedding alignment, the attention map, and the subject identity in different training stages. We also introduce a cross-attention map regularization term to enhance the learning of the attention map. Our method demonstrates significant improvements in identity preservation and text alignment compared to the baseline methods.