DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion

TL;DR

DiffusionPoser is a real-time, flexible human motion reconstruction model that uses a single diffusion process to accurately infer full-body motion from arbitrary sparse sensors without retraining.

Contribution

It introduces a novel autoregressive diffusion model capable of real-time human motion reconstruction from any sensor configuration, offering flexibility and realism without retraining.

Findings

Achieves real-time motion reconstruction from arbitrary sensor setups

Provides realistic motion outputs even for unmeasured degrees-of-freedom

Operates without retraining for different sensor arrangements

Abstract

Motion capture from a limited number of body-worn sensors, such as inertial measurement units (IMUs) and pressure insoles, has important applications in health, human performance, and entertainment. Recent work has focused on accurately reconstructing whole-body motion from a specific sensor configuration using six IMUs. While a common goal across applications is to use the minimal number of sensors to achieve required accuracy, the optimal arrangement of the sensors might differ from application to application. We propose a single diffusion model, DiffusionPoser, which reconstructs human motion in real-time from an arbitrary combination of sensors, including IMUs placed at specified locations, and, pressure insoles. Unlike existing methods, our model grants users the flexibility to determine the number and arrangement of sensors tailored to the specific activity of interest, without the need for retraining. A novel autoregressive inferencing scheme ensures real-time motion reconstruction that closely aligns with measured sensor signals. The generative nature of DiffusionPoser ensures realistic behavior, even for degrees-of-freedom not directly measured. Qualitative results can be found on our website: https://diffusionposer.github.io/.