PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium

TL;DR

PersonaMagic introduces a stage-regulated, high-fidelity face customization method that balances concept reconstruction and editability by leveraging stage partitioning and a Tandem Equilibrium mechanism, outperforming existing techniques.

Contribution

The paper proposes a novel stage-regulated generative approach with Tandem Equilibrium for improved face customization and concept balancing in personalized image generation.

Findings

Outperforms state-of-the-art methods in face customization quality.

Demonstrates robustness and flexibility across non-facial domains.

Enhances pretrained personalization models as a plug-in.

Abstract

Personalized image generation has made significant strides in adapting content to novel concepts. However, a persistent challenge remains: balancing the accurate reconstruction of unseen concepts with the need for editability according to the prompt, especially when dealing with the complex nuances of facial features. In this study, we delve into the temporal dynamics of the text-to-image conditioning process, emphasizing the crucial role of stage partitioning in introducing new concepts. We present PersonaMagic, a stage-regulated generative technique designed for high-fidelity face customization. Using a simple MLP network, our method learns a series of embeddings within a specific timestep interval to capture face concepts. Additionally, we develop a Tandem Equilibrium mechanism that adjusts self-attention responses in the text encoder, balancing text description and identity preservation, improving both areas. Extensive experiments confirm the superiority of PersonaMagic over state-of-the-art methods in both qualitative and quantitative evaluations. Moreover, its robustness and flexibility are validated in non-facial domains, and it can also serve as a valuable plug-in for enhancing the performance of pretrained personalization models.