AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation

TL;DR

AnyCrowd introduces a novel diffusion transformer framework with instance-isolated encoding and decoupled attention to enable scalable, multi-character animation with improved identity control and reduced entanglement.

Contribution

The paper presents a new framework combining instance-isolated latent representations and decoupled attention mechanisms for multi-character animation.

Findings

Successfully scales to arbitrary number of characters.

Reduces identity entanglement and bleeding.

Achieves spatio-temporally consistent animations.

Abstract

Controllable character animation has advanced rapidly in recent years, yet multi-character animation remains underexplored. As the number of characters grows, multi-character reference encoding becomes more susceptible to latent identity entanglement, resulting in identity bleeding and reduced controllability. Moreover, learning precise and spatio-temporally consistent correspondences between reference identities and driving pose sequences becomes increasingly challenging, often leading to identity-pose mis-binding and inconsistency in generated videos. To address these challenges, we propose AnyCrowd, a Diffusion Transformer (DiT)-based video generation framework capable of scaling to an arbitrary number of characters. Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement. Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global foreground-background coordination. Furthermore, to mitigate token ambiguity in overlapping regions, an Adaptive Gated Fusion (AGF) module is integrated within TSDA to predict identity-aware weights, effectively fusing competing token groups into identity-consistent representations...