Pressure2Motion: Hierarchical Human Motion Reconstruction from Ground Pressure with Text Guidance

TL;DR

Pressure2Motion introduces a novel, privacy-preserving motion capture method that reconstructs human motion from ground pressure data guided by text prompts, using hierarchical diffusion models to achieve high-fidelity results.

Contribution

It is the first to combine pressure data and linguistic priors for human motion reconstruction, establishing a new benchmark and state-of-the-art performance.

Findings

High-fidelity, physically plausible motions generated

Outperforms existing methods on the MPL benchmark

First to leverage pressure data with text guidance for motion capture

Abstract

We present Pressure2Motion, a novel motion capture algorithm that reconstructs human motion from a ground pressure sequence and text prompt. At inference time, Pressure2Motion requires only a pressure mat, eliminating the need for specialized lighting setups, cameras, or wearable devices, making it suitable for privacy-preserving, low-light, and low-cost motion capture scenarios. Such a task is severely ill-posed due to the indeterminacy of pressure signals with respect to full-body motion. To address this issue, we introduce Pressure2Motion, a generative model that leverages pressure features as input and utilizes a text prompt as a high-level guiding constraint to resolve ambiguities. Specifically, our model adopts a dual-level feature extractor to accurately interpret pressure data, followed by a hierarchical diffusion model that discerns broad-scale movement trajectories and subtle posture adjustments. Both the physical cues gained from the pressure sequence and the semantic guidance derived from descriptive texts are leveraged to guide the motion estimation with precision. To the best of our knowledge, Pressure2Motion is a pioneering work in leveraging both pressure data and linguistic priors for motion reconstruction, and the established MPL benchmark is the first benchmark for this novel motion capture task. Experiments show that our method generates high-fidelity, physically plausible motions, establishing a new state of the art for this task. The codes and benchmarks will be publicly released upon publication.