ChoreoGraph: Music-conditioned Automatic Dance Choreography over a Style and Tempo Consistent Dynamic Graph

TL;DR

ChoreoGraph is a data-driven framework that generates aesthetically consistent, music-aligned dance motions over a dynamic graph, effectively capturing style, tempo, and rhythmic nuances for diverse, high-quality choreography.

Contribution

The paper introduces ChoreoGraph, a novel dynamic graph-based approach that integrates style, tempo, and rhythmic alignment for automatic dance choreography conditioned on music.

Findings

Outperforms baseline models in quantitative metrics

Generates diverse and aesthetically consistent dance motions

Successfully aligns dance beats with musical features

Abstract

To generate dance that temporally and aesthetically matches the music is a challenging problem, as the following factors need to be considered. First, the aesthetic styles and messages conveyed by the motion and music should be consistent. Second, the beats of the generated motion should be locally aligned to the musical features. And finally, basic choreomusical rules should be observed, and the motion generated should be diverse. To address these challenges, we propose ChoreoGraph, which choreographs high-quality dance motion for a given piece of music over a Dynamic Graph. A data-driven learning strategy is proposed to evaluate the aesthetic style and rhythmic connections between music and motion in a progressively learned cross-modality embedding space. The motion sequences will be beats-aligned based on the music segments and then incorporated as nodes of a Dynamic Motion Graph. Compatibility factors such as the style and tempo consistency, motion context connection, action completeness, and transition smoothness are comprehensively evaluated to determine the node transition in the graph. We demonstrate that our repertoire-based framework can generate motions with aesthetic consistency and robustly extensible in diversity. Both quantitative and qualitative experiment results show that our proposed model outperforms other baseline models.