A Self-Supervised Approach on Motion Calibration for Enhancing Physical Plausibility in Text-to-Motion

TL;DR

This paper presents DMC, a self-supervised post-hoc module that refines text-to-motion generated human motions, improving physical realism such as foot contact and reducing artifacts, without complex physical modeling.

Contribution

Introduction of DMC, a self-supervised, data-driven motion calibrator that enhances physical plausibility in text-to-motion generation models without requiring physical simulation.

Findings

Reduces FID score by 42.74% on T2M

Improves physical plausibility by reducing foot penetration by 33%

Enhances semantic consistency and motion realism

Abstract

Generating semantically aligned human motion from textual descriptions has made rapid progress, but ensuring both semantic and physical realism in motion remains a challenge. In this paper, we introduce the Distortion-aware Motion Calibrator (DMC), a post-hoc module that refines physically implausible motions (e.g., foot floating) while preserving semantic consistency with the original textual description. Rather than relying on complex physical modeling, we propose a self-supervised and data-driven approach, whereby DMC learns to obtain physically plausible motions when an intentionally distorted motion and the original textual descriptions are given as inputs. We evaluate DMC as a post-hoc module to improve motions obtained from various text-to-motion generation models and demonstrate its effectiveness in improving physical plausibility while enhancing semantic consistency. The experimental results show that DMC reduces FID score by 42.74% on T2M and 13.20% on T2M-GPT, while also achieving the highest R-Precision. When applied to high-quality models like MoMask, DMC improves the physical plausibility of motions by reducing penetration by 33.0% as well as adjusting floating artifacts closer to the ground-truth reference. These results highlight that DMC can serve as a promising post-hoc motion refinement framework for any kind of text-to-motion models by incorporating textual semantics and physical plausibility.