Dance Generation by Sound Symbolic Words

TL;DR

This paper presents a novel method for generating dance motions from sound-symbolic words like onomatopoeia, leveraging a new dataset and adapting existing frameworks to enable intuitive, culturally diverse dance creation without requiring specialized knowledge.

Contribution

It introduces a new approach using onomatopoeia for dance generation, including a dataset and adaptation of the AI Choreographer framework for sound-symbolic input.

Findings

Enables intuitive dance generation from onomatopoeia

Creates diverse dance motions across languages

Demonstrates effectiveness through qualitative samples

Abstract

This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https://sites.google.com/view/onomatopoeia-dance/home/.