DanceCrafter: Fine-Grained Text-Driven Controllable Dance Generation via Choreographic Syntax

TL;DR

DanceCrafter introduces a novel framework and dataset for fine-grained, text-driven controllable dance generation, leveraging choreographic syntax and a specialized motion transformer.

Contribution

It proposes Choreographic Syntax for detailed dance annotation, constructs DanceFlow dataset with 41 hours of motion and descriptions, and develops DanceCrafter, a tailored motion transformer for high-quality dance synthesis.

Findings

Achieves state-of-the-art motion quality and controllability.

Demonstrates high-fidelity and stable complex dance sequence generation.

User studies confirm naturalness and precision of generated dances.

Abstract

Text-driven controllable dance generation remains under-explored, primarily due to the severe scarcity of high-quality datasets and the inherent difficulty of articulating complex choreographies. Characterizing dance is particularly challenging owing to its intricate spatial dynamics, strong directionality, and the highly decoupled movements of distinct body parts. To overcome these bottlenecks, we bridge principles from dance studies, human anatomy, and biomechanics to propose \textit{Choreographic Syntax}, a novel theoretical framework with a tailored annotation system. Grounded in this syntax, we combine professional dance archives with high-fidelity motion capture data to construct \textbf{DanceFlow}, the most fine-grained dance dataset to date. It encompasses 41 hours of high-quality motions paired with 6.34 million words of detailed descriptions. At the model level, we introduce \textbf{DanceCrafter}, a tailored motion transformer built upon the Momentum Human Rig. To circumvent optimization instabilities, we construct a continuous manifold motion representation paired with a hybrid normalization strategy. Furthermore, we design an anatomy-aware loss to explicitly regulate the decoupled nature of body parts. Together, these adaptations empower DanceCrafter to achieve the high-fidelity and stable generation of complex dance sequences. Extensive evaluations and user studies demonstrate our state-of-the-art performance in motion quality, fine-grained controllability, and generation naturalness.