Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Models

TL;DR

This paper introduces GaIP, a novel diffusion model-based approach for estimating full-body human poses from sparse, loosely attached inertial sensors, addressing real-world variability in sensor placement.

Contribution

The paper presents a garment-aware diffusion model that improves pose estimation accuracy from loose IMU data, a significant advancement over existing tightly-attached sensor methods.

Findings

Outperforms state-of-the-art inertial pose estimators quantitatively.

Synthesizes realistic loose IMU data using transformer-based diffusion models.

Enhances pose estimation robustness by incorporating garment-related parameters.

Abstract

Motion capture using sparse inertial sensors has shown great promise due to its portability and lack of occlusion issues compared to camera-based tracking. Existing approaches typically assume that IMU sensors are tightly attached to the human body. However, this assumption often does not hold in real-world scenarios. In this paper, we present Garment Inertial Poser (GaIP), a method for estimating full-body poses from sparse and loosely attached IMU sensors. We first simulate IMU recordings using an existing garment-aware human motion dataset. Our transformer-based diffusion models synthesize loose IMU data and estimate human poses from this challenging loose IMU data. We also demonstrate that incorporating garment-related parameters during training on loose IMU data effectively maintains expressiveness and enhances the ability to capture variations introduced by looser or tighter garments. Our experiments show that our diffusion methods trained on simulated and synthetic data outperform state-of-the-art inertial full-body pose estimators, both quantitatively and qualitatively, opening up a promising direction for future research on motion capture from such realistic sensor placements.