Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects

TL;DR

This paper introduces LCP-Diffusion, a novel diffusion model that enables precise layout control and subject identity preservation in text-to-image generation, allowing users to create diverse images with dynamic subject features.

Contribution

The work presents a tuning-free framework with a dual layout control mechanism and a visual refining module, enhancing layout flexibility and fidelity in personalized image synthesis.

Findings

Outperforms existing methods in identity preservation and layout control

Demonstrates robust spatial control during training and inference

Enables creation of diverse images with dynamic subject features

Abstract

Diffusion models have significantly advanced text-to-image generation, laying the foundation for the development of personalized generative frameworks. However, existing methods lack precise layout controllability and overlook the potential of dynamic features of reference subjects in improving fidelity. In this work, we propose Layout-Controllable Personalized Diffusion (LCP-Diffusion) model, a novel framework that integrates subject identity preservation with flexible layout guidance in a tuning-free approach. Our model employs a Dynamic-Static Complementary Visual Refining module to comprehensively capture the intricate details of reference subjects, and introduces a Dual Layout Control mechanism to enforce robust spatial control across both training and inference stages. Extensive experiments validate that LCP-Diffusion excels in both identity preservation and layout controllability. To the best of our knowledge, this is a pioneering work enabling users to "create anything anywhere".