ReMP: Reusable Motion Prior for Multi-domain 3D Human Pose Estimation and Motion Inbetweening

TL;DR

ReMP introduces a versatile, learned motion prior that enhances 3D human pose estimation and motion inbetweening across multiple sensor modalities, handling occlusions and noisy data effectively.

Contribution

The paper proposes ReMP, a novel reusable motion prior that captures 3D human motion dynamics and improves pose estimation in diverse, challenging scenarios.

Findings

ReMP outperforms baseline methods on various 3D motion datasets.

It effectively handles occlusions and noisy measurements.

It accelerates training for mesh sequence recovery.

Abstract

We present Reusable Motion prior (ReMP), an effective motion prior that can accurately track the temporal evolution of motion in various downstream tasks. Inspired by the success of foundation models, we argue that a robust spatio-temporal motion prior can encapsulate underlying 3D dynamics applicable to various sensor modalities. We learn the rich motion prior from a sequence of complete parametric models of posed human body shape. Our prior can easily estimate poses in missing frames or noisy measurements despite significant occlusion by employing a temporal attention mechanism. More interestingly, our prior can guide the system with incomplete and challenging input measurements to quickly extract critical information to estimate the sequence of poses, significantly improving the training efficiency for mesh sequence recovery. ReMP consistently outperforms the baseline method on diverse and practical 3D motion data, including depth point clouds, LiDAR scans, and IMU sensor data. Project page is available in https://hojunjang17.github.io/ReMP.