FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation

TL;DR

This paper introduces FineDance, a comprehensive dataset with 14.6 hours of music-dance pairs across 22 genres, and a novel dance generation network that produces realistic, genre-coherent 3D full-body dance sequences using diffusion models and retrieval modules.

Contribution

The paper presents the largest multi-genre dance dataset and a new full-body dance generation network leveraging diffusion models and retrieval modules for improved realism and genre consistency.

Findings

FineDance contains 14.6 hours of data with 22 dance genres.

The proposed FineNet achieves state-of-the-art dance generation quality.

The Genre&Coherent aware Retrieval Module enhances genre matching and stability.

Abstract

Generating full-body and multi-genre dance sequences from given music is a challenging task, due to the limitations of existing datasets and the inherent complexity of the fine-grained hand motion and dance genres. To address these problems, we propose FineDance, which contains 14.6 hours of music-dance paired data, with fine-grained hand motions, fine-grained genres (22 dance genres), and accurate posture. To the best of our knowledge, FineDance is the largest music-dance paired dataset with the most dance genres. Additionally, to address monotonous and unnatural hand movements existing in previous methods, we propose a full-body dance generation network, which utilizes the diverse generation capabilities of the diffusion model to solve monotonous problems, and use expert nets to solve unreal problems. To further enhance the genre-matching and long-term stability of generated dances, we propose a Genre&Coherent aware Retrieval Module. Besides, we propose a novel metric named Genre Matching Score to evaluate the genre-matching degree between dance and music. Quantitative and qualitative experiments demonstrate the quality of FineDance, and the state-of-the-art performance of FineNet. The FineDance Dataset and more qualitative samples can be found at our website.