EverybodyDance: Bipartite Graph-Based Identity Correspondence for Multi-Character Animation

TL;DR

EverybodyDance introduces a bipartite graph-based method with a novel matching metric and strategies to ensure correct identity correspondence in multi-character pose-driven animation, significantly improving over existing methods.

Contribution

The paper presents a new framework with the Identity Matching Graph and Mask-Query Attention for enforcing identity correspondence in multi-character animation, addressing a key challenge in the field.

Findings

Outperforms state-of-the-art in identity correspondence accuracy

Improves visual fidelity in multi-character animations

Provides a new benchmark for multi-character identity correspondence

Abstract

Consistent pose-driven character animation has achieved remarkable progress in single-character scenarios. However, extending these advances to multi-character settings is non-trivial, especially when position swap is involved. Beyond mere scaling, the core challenge lies in enforcing correct Identity Correspondence (IC) between characters in reference and generated frames. To address this, we introduce EverybodyDance, a systematic solution targeting IC correctness in multi-character animation. EverybodyDance is built around the Identity Matching Graph (IMG), which models characters in the generated and reference frames as two node sets in a weighted complete bipartite graph. Edge weights, computed via our proposed Mask-Query Attention (MQA), quantify the affinity between each pair of characters. Our key insight is to formalize IC correctness as a graph structural metric and to optimize it during training. We also propose a series of targeted strategies tailored for multi-character animation, including identity-embedded guidance, a multi-scale matching strategy, and pre-classified sampling, which work synergistically. Finally, to evaluate IC performance, we curate the Identity Correspondence Evaluation benchmark, dedicated to multi-character IC correctness. Extensive experiments demonstrate that EverybodyDance substantially outperforms state-of-the-art baselines in both IC and visual fidelity.