Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matching

TL;DR

This paper introduces a novel diffusion model fine-tuned with latent concatenation and masked flow matching to improve subject identity preservation and prompt diversity in image generation, supported by a large-scale data curation and evaluation framework.

Contribution

It proposes a new latent concatenation strategy with masked flow matching for identity preservation without architectural changes, and a two-stage data curation process for scalable training.

Findings

Enhanced identity consistency in generated images.

Improved prompt diversity across subjects and contexts.

Effective large-scale training with curated datasets.

Abstract

Subject-driven image generation aims to synthesize novel depictions of a specific subject across diverse contexts while preserving its core identity features. Achieving both strong identity consistency and high prompt diversity presents a fundamental trade-off. We propose a LoRA fine-tuned diffusion model employing a latent concatenation strategy, which jointly processes reference and target images, combined with a masked Conditional Flow Matching (CFM) objective. This approach enables robust identity preservation without architectural modifications. To facilitate large-scale training, we introduce a two-stage Distilled Data Curation Framework: the first stage leverages data restoration and VLM-based filtering to create a compact, high-quality seed dataset from diverse sources; the second stage utilizes these curated examples for parameter-efficient fine-tuning, thus scaling the generation capability across various subjects and contexts. Finally, for filtering and quality assessment, we present CHARIS, a fine-grained evaluation framework that performs attribute-level comparisons along five key axes: identity consistency, prompt adherence, region-wise color fidelity, visual quality, and transformation diversity.