Priority-Centric Human Motion Generation in Discrete Latent Space

TL;DR

This paper introduces a priority-centric diffusion model for text-to-human motion generation that emphasizes salient motions based on their importance, resulting in more semantically rich and diverse motions.

Contribution

It proposes a novel discrete diffusion model with a significance-aware noise schedule and a Transformer-based VQ-VAE for improved motion generation.

Findings

Outperforms existing methods in fidelity and diversity

Effectively captures salient motions in complex descriptions

Enhances semantic richness of generated motions

Abstract

Text-to-motion generation is a formidable task, aiming to produce human motions that align with the input text while also adhering to human capabilities and physical laws. While there have been advancements in diffusion models, their application in discrete spaces remains underexplored. Current methods often overlook the varying significance of different motions, treating them uniformly. It is essential to recognize that not all motions hold the same relevance to a particular textual description. Some motions, being more salient and informative, should be given precedence during generation. In response, we introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM), which utilizes a Transformer-based VQ-VAE to derive a concise, discrete motion representation, incorporating a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token within the entire motion sequence. This approach retains the most salient motions during the reverse diffusion process, leading to more semantically rich and varied motions. Additionally, we formulate two strategies to gauge the importance of motion tokens, drawing from both textual and visual indicators. Comprehensive experiments on the HumanML3D and KIT-ML datasets confirm that our model surpasses existing techniques in fidelity and diversity, particularly for intricate textual descriptions.