Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis

TL;DR

This paper introduces Event-T2M, a diffusion-based framework that decomposes complex multi-action text prompts into semantically self-contained events, improving the naturalness and order of generated motions, especially for multi-event prompts.

Contribution

It defines the concept of an event in text-to-motion synthesis, proposes a novel event-based conditioning method, and introduces a new benchmark for multi-event prompts to evaluate model performance.

Findings

Event-T2M outperforms baselines on multi-event prompts.

The new HumanML3D-E benchmark effectively stratifies prompts by event complexity.

Human studies confirm the naturalness and order preservation of generated motions.

Abstract

Text-to-motion generation has advanced with diffusion models, yet existing systems often collapse complex multi-action prompts into a single embedding, leading to omissions, reordering, or unnatural transitions. In this work, we shift perspective by introducing a principled definition of an event as the smallest semantically self-contained action or state change in a text prompt that can be temporally aligned with a motion segment. Building on this definition, we propose Event-T2M, a diffusion-based framework that decomposes prompts into events, encodes each with a motion-aware retrieval model, and integrates them through event-based cross-attention in Conformer blocks. Existing benchmarks mix simple and multi-event prompts, making it unclear whether models that succeed on single actions generalize to multi-action cases. To address this, we construct HumanML3D-E, the first benchmark stratified by event count. Experiments on HumanML3D, KIT-ML, and HumanML3D-E show that Event-T2M matches state-of-the-art baselines on standard tests while outperforming them as event complexity increases. Human studies validate the plausibility of our event definition, the reliability of HumanML3D-E, and the superiority of Event-T2M in generating multi-event motions that preserve order and naturalness close to ground-truth. These results establish event-level conditioning as a generalizable principle for advancing text-to-motion generation beyond single-action prompts.