MUSAR: Exploring Multi-Subject Customization from Single-Subject Dataset via Attention Routing

TL;DR

MUSAR is a novel framework that enables multi-subject image customization using only single-subject data by employing attention routing and diptych learning to address data scarcity and attribute entanglement.

Contribution

The paper introduces debiased diptych learning and dynamic attention routing to achieve multi-subject customization from single-subject datasets, overcoming key limitations of existing methods.

Findings

Outperforms existing multi-subject methods in image quality and consistency

Effective decoupling of multi-subject representations

Maintains scalability with increasing reference subjects

Abstract

Current multi-subject customization approaches encounter two critical challenges: the difficulty in acquiring diverse multi-subject training data, and attribute entanglement across different subjects. To bridge these gaps, we propose MUSAR - a simple yet effective framework to achieve robust multi-subject customization while requiring only single-subject training data. Firstly, to break the data limitation, we introduce debiased diptych learning. It constructs diptych training pairs from single-subject images to facilitate multi-subject learning, while actively correcting the distribution bias introduced by diptych construction via static attention routing and dual-branch LoRA. Secondly, to eliminate cross-subject entanglement, we introduce dynamic attention routing mechanism, which adaptively establishes bijective mappings between generated images and conditional subjects. This design not only achieves decoupling of multi-subject representations but also maintains scalable generalization performance with increasing reference subjects. Comprehensive experiments demonstrate that our MUSAR outperforms existing methods - even those trained on multi-subject dataset - in image quality, subject consistency, and interaction naturalness, despite requiring only single-subject dataset.