LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation

TL;DR

LumosX introduces a novel framework combining data collection and advanced attention mechanisms to improve identity consistency and attribute alignment in personalized multi-subject video generation using diffusion models.

Contribution

It proposes a comprehensive approach with a new data pipeline and relational attention modules to enhance face-attribute alignment and intra-group consistency in personalized videos.

Findings

Achieves state-of-the-art results in personalized multi-subject video generation.

Demonstrates improved identity consistency and semantic alignment.

Provides a new benchmark for evaluating personalized video generation methods.

Abstract

Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.