MC$^2$: Multi-concept Guidance for Customized Multi-concept Generation

TL;DR

MC$^2$ introduces an inference-time optimization method for multi-concept text-to-image generation, improving integration and fidelity of multiple concepts without retraining models, and offers a new benchmark for evaluation.

Contribution

The paper presents MC$^2$, a novel inference-time approach that enables flexible multi-concept customization in text-to-image generation, handling heterogeneous models and improving concept integration.

Findings

Outperforms training-based methods in prompt-reference alignment.

Enables robust compositional text-to-image generation.

Introduces the MC++ benchmark for multi-concept evaluation.

Abstract

Customized text-to-image generation, which synthesizes images based on user-specified concepts, has made significant progress in handling individual concepts. However, when extended to multiple concepts, existing methods often struggle with properly integrating different models and avoiding the unintended blending of characteristics from distinct concepts. In this paper, we propose MC$^2$, a novel approach for multi-concept customization that enhances flexibility and fidelity through inference-time optimization. MC$^2$ enables the integration of multiple single-concept models with heterogeneous architectures. By adaptively refining attention weights between visual and textual tokens, our method ensures that image regions accurately correspond to their associated concepts while minimizing interference between concepts. Extensive experiments demonstrate that MC$^2$ outperforms training-based methods in terms of prompt-reference alignment. Furthermore, MC$^2$ can be seamlessly applied to text-to-image generation, providing robust compositional capabilities. To facilitate the evaluation of multi-concept customization, we also introduce a new benchmark, MC++. The code will be publicly available at https://github.com/JIANGJiaXiu/MC-2.