3D Human Pose Estimation via Intuitive Physics

TL;DR

This paper introduces IPMAN, a novel method for 3D human pose estimation that incorporates intuitive-physics constraints derived from biomechanics to produce more physically plausible and accurate results.

Contribution

We propose a differentiable intuitive-physics framework that infers pressure heatmaps, CoP, and CoM from images to improve 3D human pose estimation.

Findings

IPMAN outperforms state-of-the-art methods on standard datasets.

The approach improves plausibility and accuracy of static pose estimations.

It maintains performance on dynamic poses without degradation.

Abstract

Estimating 3D humans from images often produces implausible bodies that lean, float, or penetrate the floor. Such methods ignore the fact that bodies are typically supported by the scene. A physics engine can be used to enforce physical plausibility, but these are not differentiable, rely on unrealistic proxy bodies, and are difficult to integrate into existing optimization and learning frameworks. In contrast, we exploit novel intuitive-physics (IP) terms that can be inferred from a 3D SMPL body interacting with the scene. Inspired by biomechanics, we infer the pressure heatmap on the body, the Center of Pressure (CoP) from the heatmap, and the SMPL body's Center of Mass (CoM). With these, we develop IPMAN, to estimate a 3D body from a color image in a "stable" configuration by encouraging plausible floor contact and overlapping CoP and CoM. Our IP terms are intuitive, easy to implement, fast to compute, differentiable, and can be integrated into existing optimization and regression methods. We evaluate IPMAN on standard datasets and MoYo, a new dataset with synchronized multi-view images, ground-truth 3D bodies with complex poses, body-floor contact, CoM and pressure. IPMAN produces more plausible results than the state of the art, improving accuracy for static poses, while not hurting dynamic ones. Code and data are available for research at https://ipman.is.tue.mpg.de.