FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling

TL;DR

FlowCoMotion introduces a unified token-latent flow modeling framework for text-to-motion generation, effectively capturing semantic content and motion details through combined continuous and discrete representations.

Contribution

It proposes a novel token-latent coupling approach that unifies continuous and discrete motion representations for improved text-to-motion synthesis.

Findings

Achieves competitive results on HumanML3D and SnapMoGen benchmarks.

Effectively captures both semantic and fine-grained motion details.

Outperforms existing methods in text-to-motion generation tasks.

Abstract

Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.