MotionHiFlow: Text-to-motion via hierarchical flow matching

TL;DR

MotionHiFlow introduces a hierarchical flow matching framework for text-to-motion generation, capturing multi-scale motion semantics and details to produce more coherent and semantically aligned 3D human motions.

Contribution

The paper proposes a novel hierarchical flow matching approach that models motions across multiple temporal scales, improving semantic alignment and motion detail in text-to-motion generation.

Findings

Achieves state-of-the-art results on HumanML3D and KIT-ML benchmarks.

Demonstrates the effectiveness of hierarchical design through ablation studies.

Integrates a Text-Motion Diffusion Transformer and topology-aware Motion VAE for better structural modeling.

Abstract

Text-to-motion generation aims to generate 3D human motions that are tightly aligned with the input text while remaining physically plausible and rich in fine-grained detail. Although recent approaches can produce complex and natural movements, they usually operate at only one temporal scale, which limits both semantic alignment and temporal coherence. Inspired by the fact that complex motions are conceptualized hierarchically rather than at a single temporal scale in the human cognitive system, we propose \textit{MotionHiFlow}, a hierarchical flow matching framework to generate motion progressively by constructing flow path from low to high temporal scales. The flows at lower scales capture high-level semantics and coarse motion structures, while flows at higher scales refine temporal details. To link the flows across scales, we introduce a novel cross-scale transition process, ensuring continuity and preserving noise consistency. Furthermore, by integrating a Text-Motion Diffusion Transformer and a topology-aware Motion VAE, MotionHiFlow explicitly models structural dependencies among joints via joint-aware positional encoding and skeletal topology, enabling precise semantic alignment alongside fine-grained motion details. Extensive experiments on HumanML3D and KIT-ML benchmarks demonstrate state-of-the-art performance, with ablation studies confirming the effectiveness of the hierarchical design and key components. Code is available at https://github.com/ai-lh/MotionHiFlow.