Dancing to Music
Hsin-Ying Lee, Xiaodong Yang, Ming-Yu Liu, Ting-Chun Wang, Yu-Ding Lu,, Ming-Hsuan Yang, Jan Kautz

TL;DR
This paper introduces a novel framework for generating dance movements from music by analyzing dance into basic units and synthesizing them based on musical input, capturing style and beat.
Contribution
It presents a new synthesis-by-analysis approach that decomposes dance into basic units and learns to compose realistic dances aligned with music.
Findings
Synthesizes realistic and diverse dance movements
Ensures dance style consistency with music
Matches dance movements to musical beats
Abstract
Dancing to music is an instinctive move by humans. Learning to model the music-to-dance generation process is, however, a challenging problem. It requires significant efforts to measure the correlation between music and dance as one needs to simultaneously consider multiple aspects, such as style and beat of both music and dance. Additionally, dance is inherently multimodal and various following movements of a pose at any moment are equally likely. In this paper, we propose a synthesis-by-analysis learning framework to generate dance from music. In the analysis phase, we decompose a dance into a series of basic dance units, through which the model learns how to move. In the synthesis phase, the model learns how to compose a dance by organizing multiple basic dancing movements seamlessly according to the input music. Experimental qualitative and quantitative results demonstrate that the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · Music Technology and Sound Studies · Music and Audio Processing
