MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis

TL;DR

MoFusion is a novel denoising diffusion framework for human motion synthesis that produces high-quality, diverse, and semantically accurate motions conditioned on various inputs, with applications in editing and animation.

Contribution

It introduces MoFusion, a diffusion-based approach that enhances motion diversity and quality, integrating kinematic losses and enabling interactive editing capabilities.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Generates long, plausible, and semantically accurate motions.

Enables interactive motion editing applications.

Abstract

Conventional methods for human motion synthesis are either deterministic or struggle with the trade-off between motion diversity and motion quality. In response to these limitations, we introduce MoFusion, i.e., a new denoising-diffusion-based framework for high-quality conditional human motion synthesis that can generate long, temporally plausible, and semantically accurate motions based on a range of conditioning contexts (such as music and text). We also present ways to introduce well-known kinematic losses for motion plausibility within the motion diffusion framework through our scheduled weighting strategy. The learned latent space can be used for several interactive motion editing applications -- like inbetweening, seed conditioning, and text-based editing -- thus, providing crucial abilities for virtual character animation and robotics. Through comprehensive quantitative evaluations and a perceptual user study, we demonstrate the effectiveness of MoFusion compared to the state of the art on established benchmarks in the literature. We urge the reader to watch our supplementary video and visit https://vcai.mpi-inf.mpg.de/projects/MoFusion.