Motion Manifold Flow Primitives for Task-Conditioned Trajectory Generation under Complex Task-Motion Dependencies

TL;DR

This paper introduces Motion Manifold Flow Primitives (MMFP), a novel framework that decouples motion manifold learning from task conditioning, enabling effective generation of complex, task-dependent trajectories from human demonstrations.

Contribution

The paper proposes MMFP, which uses flow matching models to learn task-conditioned distributions in a learned motion manifold, improving trajectory generation under complex task dependencies.

Findings

MMFP outperforms existing methods in language-guided trajectory tasks.

Decoupling motion manifold learning from task conditioning enhances generative flexibility.

Experiments demonstrate superior handling of many-to-many text-motion mappings.

Abstract

Effective movement primitives should be capable of encoding and generating a rich repertoire of trajectories -- typically collected from human demonstrations -- conditioned on task-defining parameters such as vision or language inputs. While recent methods based on the motion manifold hypothesis, which assumes that a set of trajectories lies on a lower-dimensional nonlinear subspace, address challenges such as limited dataset size and the high dimensionality of trajectory data, they often struggle to capture complex task-motion dependencies, i.e., when motion distributions shift drastically with task variations. To address this, we introduce Motion Manifold Flow Primitives (MMFP), a framework that decouples the training of the motion manifold from task-conditioned distributions. Specifically, we employ flow matching models, state-of-the-art conditional deep generative models, to learn task-conditioned distributions in the latent coordinate space of the learned motion manifold. Experiments are conducted on language-guided trajectory generation tasks, where many-to-many text-motion correspondences introduce complex task-motion dependencies, highlighting MMFP's superiority over existing methods.