CoDi: Subject-Consistent and Pose-Diverse Text-to-Image Generation

TL;DR

CoDi introduces a two-stage diffusion-based framework that achieves subject consistency and pose diversity in text-to-image generation, enhancing visual storytelling without sacrificing layout variety.

Contribution

It presents a novel two-stage method using optimal transport and feature refinement to improve subject consistency and pose diversity in T2I models.

Findings

Outperforms existing methods in subject consistency and pose diversity.

Maintains high prompt fidelity and visual quality.

Demonstrates effectiveness through extensive qualitative and quantitative evaluations.

Abstract

Subject-consistent generation (SCG)-aiming to maintain a consistent subject identity across diverse scenes-remains a challenge for text-to-image (T2I) models. Existing training-free SCG methods often achieve consistency at the cost of layout and pose diversity, hindering expressive visual storytelling. To address the limitation, we propose subject-Consistent and pose-Diverse T2I framework, dubbed as CoDi, that enables consistent subject generation with diverse pose and layout. Motivated by the progressive nature of diffusion, where coarse structures emerge early and fine details are refined later, CoDi adopts a two-stage strategy: Identity Transport (IT) and Identity Refinement (IR). IT operates in the early denoising steps, using optimal transport to transfer identity features to each target image in a pose-aware manner. This promotes subject consistency while preserving pose diversity. IR is applied in the later denoising steps, selecting the most salient identity features to further refine subject details. Extensive qualitative and quantitative results on subject consistency, pose diversity, and prompt fidelity demonstrate that CoDi achieves both better visual perception and stronger performance across all metrics. The code is provided in https://github.com/NJU-PCALab/CoDi.