GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation

TL;DR

GCDance is a diffusion-based framework that generates genre-specific 3D dance sequences synchronized with music and guided by descriptive text, ensuring controllability and realism.

Contribution

It introduces a novel text-based control mechanism and multi-task optimization to improve genre control and alignment in music-driven dance generation.

Findings

Outperforms state-of-the-art methods on FineDance and AIST++ datasets.

Effectively incorporates genre and music features for coherent dance synthesis.

Balances realism, accuracy, and style through multi-task optimization.

Abstract

Music-driven dance generation is a challenging task as it requires strict adherence to genre-specific choreography while ensuring physically realistic and precisely synchronized dance sequences with the music's beats and rhythm. Although significant progress has been made in music-conditioned dance generation, most existing methods struggle to convey specific stylistic attributes in generated dance. To bridge this gap, we propose a diffusion-based framework for genre-specific 3D full-body dance generation, conditioned on both music and descriptive text. To effectively incorporate genre information, we develop a text-based control mechanism that maps input prompts, either explicit genre labels or free-form descriptive text, into genre-specific control signals, enabling precise and controllable text-guided generation of genre-consistent dance motions. Furthermore, to enhance the alignment between music and textual conditions, we leverage the features of a music foundation model, facilitating coherent and semantically aligned dance synthesis. Last, to balance the objectives of extracting text-genre information and maintaining high-quality generation results, we propose a novel multi-task optimization strategy. This effectively balances competing factors such as physical realism, spatial accuracy, and text classification, significantly improving the overall quality of the generated sequences. Extensive experimental results obtained on the FineDance and AIST++ datasets demonstrate the superiority of GCDance over the existing state-of-the-art approaches.