Length-Aware Motion Synthesis via Latent Diffusion

TL;DR

This paper introduces LADiff, a novel length-aware latent diffusion model that synthesizes 3D human motions of variable lengths from text, improving control and detail over prior methods.

Contribution

The paper proposes a new length-aware variational auto-encoder and a length-conforming latent diffusion model for more accurate and controllable motion synthesis.

Findings

LADiff outperforms state-of-the-art methods on HumanML3D and KIT-ML benchmarks.

It provides better control over motion duration and style.

The model generates more detailed motions for longer sequences.

Abstract

The target duration of a synthesized human motion is a critical attribute that requires modeling control over the motion dynamics and style. Speeding up an action performance is not merely fast-forwarding it. However, state-of-the-art techniques for human behavior synthesis have limited control over the target sequence length. We introduce the problem of generating length-aware 3D human motion sequences from textual descriptors, and we propose a novel model to synthesize motions of variable target lengths, which we dub "Length-Aware Latent Diffusion" (LADiff). LADiff consists of two new modules: 1) a length-aware variational auto-encoder to learn motion representations with length-dependent latent codes; 2) a length-conforming latent diffusion model to generate motions with a richness of details that increases with the required target sequence length. LADiff significantly improves over the state-of-the-art across most of the existing motion synthesis metrics on the two established benchmarks of HumanML3D and KIT-ML.