Guided Motion Diffusion for Controllable Human Motion Synthesis

TL;DR

This paper introduces Guided Motion Diffusion, a novel method that incorporates spatial constraints into human motion synthesis using diffusion models, improving control and realism in generated motions.

Contribution

The paper proposes a new feature projection scheme and dense guidance approach to effectively integrate spatial constraints into diffusion-based human motion synthesis.

Findings

Significant improvement over state-of-the-art in text-based motion generation

Effective incorporation of spatial constraints like trajectories and obstacles

Enhanced control over generated human motions

Abstract

Denoising diffusion models have shown great promise in human motion synthesis conditioned on natural language descriptions. However, integrating spatial constraints, such as pre-defined motion trajectories and obstacles, remains a challenge despite being essential for bridging the gap between isolated human motion and its surrounding environment. To address this issue, we propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Together with a new imputation formulation, the generated motion can reliably conform to spatial constraints such as global motion trajectories. Furthermore, given sparse spatial constraints (e.g. sparse keyframes), we introduce a new dense guidance approach to turn a sparse signal, which is susceptible to being ignored during the reverse steps, into denser signals to guide the generated motion to the given constraints. Our extensive experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation while allowing control of the synthesized motions with spatial constraints.