Generative human motion mimicking through feature extraction in denoising diffusion settings

TL;DR

This paper introduces a novel AI model that generates dance movements by mimicking and creatively enhancing motion capture data, using feature extraction in a denoising diffusion framework, enabling realistic and diverse dance synthesis.

Contribution

It is the first model to utilize high-level features from single-person motion data for creative dance generation without relying on human-human interaction data.

Findings

Generated movements are diverse and realistic.

Model achieves temporal coherence in dance sequences.

Quantitative assessment shows effective feature distribution convergence.

Abstract

Recent success with large language models has sparked a new wave of verbal human-AI interaction. While such models support users in a variety of creative tasks, they lack the embodied nature of human interaction. Dance, as a primal form of human expression, is predestined to complement this experience. To explore creative human-AI interaction exemplified by dance, we build an interactive model based on motion capture (MoCap) data. It generates an artificial other by partially mimicking and also "creatively" enhancing an incoming sequence of movement data. It is the first model, which leverages single-person motion data and high level features in order to do so and, thus, it does not rely on low level human-human interaction data. It combines ideas of two diffusion models, motion inpainting, and motion style transfer to generate movement representations that are both temporally coherent and responsive to a chosen movement reference. The success of the model is demonstrated by quantitatively assessing the convergence of the feature distribution of the generated samples and the test set which serves as simulating the human performer. We show that our generations are first steps to creative dancing with AI as they are both diverse showing various deviations from the human partner while appearing realistic.