Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Sampling

TL;DR

This paper introduces a novel diffusion-based approach for generating coherent long-term human motions from text streams, addressing the limitations of existing short-term and autoregressive methods.

Contribution

It proposes a past-conditioned diffusion model with two coherent sampling techniques, enabling the generation of continuous, compositional long-term motions guided by text.

Findings

Capable of generating coherent long-term 3D human motions

Uses past inpainting and compositional transition sampling methods

Achieves improved coherence and controllability in motion synthesis

Abstract

Text-to-motion generation has gained increasing attention, but most existing methods are limited to generating short-term motions that correspond to a single sentence describing a single action. However, when a text stream describes a sequence of continuous motions, the generated motions corresponding to each sentence may not be coherently linked. Existing long-term motion generation methods face two main issues. Firstly, they cannot directly generate coherent motions and require additional operations such as interpolation to process the generated actions. Secondly, they generate subsequent actions in an autoregressive manner without considering the influence of future actions on previous ones. To address these issues, we propose a novel approach that utilizes a past-conditioned diffusion model with two optional coherent sampling methods: Past Inpainting Sampling and Compositional Transition Sampling. Past Inpainting Sampling completes subsequent motions by treating previous motions as conditions, while Compositional Transition Sampling models the distribution of the transition as the composition of two adjacent motions guided by different text prompts. Our experimental results demonstrate that our proposed method is capable of generating compositional and coherent long-term 3D human motions controlled by a user-instructed long text stream. The code is available at \href{https://github.com/yangzhao1230/PCMDM}{https://github.com/yangzhao1230/PCMDM}.