Generative Autoregressive Networks for 3D Dancing Move Synthesis from Music

TL;DR

This paper introduces a generative autoregressive framework that creates realistic 3D dance sequences from music, capable of controlling humanoid robots to dance in sync with the music.

Contribution

It presents a novel integrated system combining music encoding, pose generation, and genre classification for realistic dance synthesis from music.

Findings

Generated dance sequences exceed 5,000 frames.

The system produces human-like dance movements.

Applied to humanoid robots for music-driven dancing.

Abstract

This paper proposes a framework which is able to generate a sequence of three-dimensional human dance poses for a given music. The proposed framework consists of three components: a music feature encoder, a pose generator, and a music genre classifier. We focus on integrating these components for generating a realistic 3D human dancing move from music, which can be applied to artificial agents and humanoid robots. The trained dance pose generator, which is a generative autoregressive model, is able to synthesize a dance sequence longer than 5,000 pose frames. Experimental results of generated dance sequences from various songs show how the proposed method generates human-like dancing move to a given music. In addition, a generated 3D dance sequence is applied to a humanoid robot, showing that the proposed framework can make a robot to dance just by listening to music.