FlexMotion: Lightweight, Physics-Aware, and Controllable Human Motion Generation

TL;DR

FlexMotion is a lightweight, physics-aware human motion generation framework that combines a diffusion model with a Transformer encoder-decoder, enabling fast, realistic, and controllable motion synthesis without physics simulators.

Contribution

It introduces a novel, efficient diffusion-based framework with a multimodal Transformer for physically plausible and controllable human motion synthesis.

Findings

Achieves realistic motion with improved efficiency and control.

Outperforms existing methods in realism and physical plausibility.

Sets a new benchmark in human motion synthesis.

Abstract

Lightweight, controllable, and physically plausible human motion synthesis is crucial for animation, virtual reality, robotics, and human-computer interaction applications. Existing methods often compromise between computational efficiency, physical realism, or spatial controllability. We propose FlexMotion, a novel framework that leverages a computationally lightweight diffusion model operating in the latent space, eliminating the need for physics simulators and enabling fast and efficient training. FlexMotion employs a multimodal pre-trained Transformer encoder-decoder, integrating joint locations, contact forces, joint actuations and muscle activations to ensure the physical plausibility of the generated motions. FlexMotion also introduces a plug-and-play module, which adds spatial controllability over a range of motion parameters (e.g., joint locations, joint actuations, contact forces, and muscle activations). Our framework achieves realistic motion generation with improved efficiency and control, setting a new benchmark for human motion synthesis. We evaluate FlexMotion on extended datasets and demonstrate its superior performance in terms of realism, physical plausibility, and controllability.