MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm

TL;DR

MotionLab introduces a unified framework for human motion generation and editing, leveraging a novel Motion-Condition-Motion paradigm to enable versatile, efficient, and controllable motion tasks with improved generalization.

Contribution

The paper proposes the Motion-Condition-Motion paradigm and MotionLab framework, integrating rectified flows, a transformer, and curriculum learning for multi-task human motion generation and editing.

Findings

Effective multi-task learning and knowledge sharing across motion tasks

Strong generalization capabilities on multiple benchmarks

High inference efficiency and versatility in motion generation and editing

Abstract

Human motion generation and editing are key components of computer vision. However, current approaches in this field tend to offer isolated solutions tailored to specific tasks, which can be inefficient and impractical for real-world applications. While some efforts have aimed to unify motion-related tasks, these methods simply use different modalities as conditions to guide motion generation. Consequently, they lack editing capabilities, fine-grained control, and fail to facilitate knowledge sharing across tasks. To address these limitations and provide a versatile, unified framework capable of handling both human motion generation and editing, we introduce a novel paradigm: \textbf{Motion-Condition-Motion}, which enables the unified formulation of diverse tasks with three concepts: source motion, condition, and target motion. Based on this paradigm, we propose a unified framework, \textbf{MotionLab}, which incorporates rectified flows to learn the mapping from source motion to target motion, guided by the specified conditions. In MotionLab, we introduce the 1) MotionFlow Transformer to enhance conditional generation and editing without task-specific modules; 2) Aligned Rotational Position Encoding to guarantee the time synchronization between source motion and target motion; 3) Task Specified Instruction Modulation; and 4) Motion Curriculum Learning for effective multi-task learning and knowledge sharing across tasks. Notably, our MotionLab demonstrates promising generalization capabilities and inference efficiency across multiple benchmarks for human motion. Our code and additional video results are available at: https://diouo.github.io/motionlab.github.io/.