MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understanding

TL;DR

MotionGPT-2 is a versatile large language model that generates and understands human motions from text and multimodal inputs, overcoming previous limitations in control and task specificity.

Contribution

It introduces a unified framework that integrates multimodal control and motion generation using discretized tokens and a novel Part-Aware VQVAE for detailed motion representation.

Findings

Effective across motion generation, captioning, and completion tasks.

Outperforms existing methods in generating lifelike, controllable human motions.

Demonstrates high adaptability and fine-grained motion control.

Abstract

Generating lifelike human motions from descriptive texts has experienced remarkable research focus in the recent years, propelled by the emerging requirements of digital humans.Despite impressive advances, existing approaches are often constrained by limited control modalities, task specificity, and focus solely on body motion representations.In this paper, we present MotionGPT-2, a unified Large Motion-Language Model (LMLM) that addresses these limitations. MotionGPT-2 accommodates multiple motion-relevant tasks and supporting multimodal control conditions through pre-trained Large Language Models (LLMs). It quantizes multimodal inputs-such as text and single-frame poses-into discrete, LLM-interpretable tokens, seamlessly integrating them into the LLM's vocabulary. These tokens are then organized into unified prompts, guiding the LLM to generate motion outputs through a pretraining-then-finetuning paradigm. We also show that the proposed MotionGPT-2 is highly adaptable to the challenging 3D holistic motion generation task, enabled by the innovative motion discretization framework, Part-Aware VQVAE, which ensures fine-grained representations of body and hand movements. Extensive experiments and visualizations validate the effectiveness of our method, demonstrating the adaptability of MotionGPT-2 across motion generation, motion captioning, and generalized motion completion tasks.