Generation of Complex 3D Human Motion by Temporal and Spatial Composition of Diffusion Models

TL;DR

This paper introduces a novel method for generating realistic 3D human motions for unseen actions by decomposing complex movements into simpler ones and recombining them using diffusion models, enhancing motion synthesis flexibility.

Contribution

It proposes a new approach that decomposes complex actions into simpler motions and recombines them during inference, enabling synthesis of unseen motion classes with pre-trained diffusion models.

Findings

Successfully generates complex 3D motions unseen during training.

Outperforms state-of-the-art methods on benchmark datasets.

Operates during inference, compatible with any pre-trained diffusion model.

Abstract

In this paper, we address the challenge of generating realistic 3D human motions for action classes that were never seen during the training phase. Our approach involves decomposing complex actions into simpler movements, specifically those observed during training, by leveraging the knowledge of human motion contained in GPTs models. These simpler movements are then combined into a single, realistic animation using the properties of diffusion models. Our claim is that this decomposition and subsequent recombination of simple movements can synthesize an animation that accurately represents the complex input action. This method operates during the inference phase and can be integrated with any pre-trained diffusion model, enabling the synthesis of motion classes not present in the training data. We evaluate our method by dividing two benchmark human motion datasets into basic and complex actions, and then compare its performance against the state-of-the-art.