PartComposer: Learning and Composing Part-Level Concepts from Single-Image Examples

TL;DR

PartComposer introduces a novel framework that enables text-to-image diffusion models to learn and compose part-level concepts from single-image examples, addressing data scarcity and improving concept disentanglement.

Contribution

It proposes a dynamic data synthesis pipeline and a mutual information maximization approach for effective part-level concept learning from minimal data.

Findings

Achieves strong disentanglement of part concepts.

Enables controllable object composition from single images.

Outperforms baseline methods in concept mixing tasks.

Abstract

We present PartComposer: a framework for part-level concept learning from single-image examples that enables text-to-image diffusion models to compose novel objects from meaningful components. Existing methods either struggle with effectively learning fine-grained concepts or require a large dataset as input. We propose a dynamic data synthesis pipeline generating diverse part compositions to address one-shot data scarcity. Most importantly, we propose to maximize the mutual information between denoised latents and structured concept codes via a concept predictor, enabling direct regulation on concept disentanglement and re-composition supervision. Our method achieves strong disentanglement and controllable composition, outperforming subject and part-level baselines when mixing concepts from the same, or different, object categories.