Trajectory Optimization for Physics-Based Reconstruction of 3d Human Pose from Monocular Video

TL;DR

This paper introduces a physics-based trajectory optimization method for reconstructing realistic 3D human motion from monocular videos, capable of handling diverse real-world scenes and complex motions.

Contribution

It integrates a comprehensive physics engine into pose estimation, enabling generalization to uncontrolled scenes and complex contact scenarios without re-training.

Findings

Achieves competitive results on Human3.6M benchmark.

Generalizes to complex motions in AIST benchmark.

Applicable to uncontrolled internet videos.

Abstract

We focus on the task of estimating a physically plausible articulated human motion from monocular video. Existing approaches that do not consider physics often produce temporally inconsistent output with motion artifacts, while state-of-the-art physics-based approaches have either been shown to work only in controlled laboratory conditions or consider simplified body-ground contact limited to feet. This paper explores how these shortcomings can be addressed by directly incorporating a fully-featured physics engine into the pose estimation process. Given an uncontrolled, real-world scene as input, our approach estimates the ground-plane location and the dimensions of the physical body model. It then recovers the physical motion by performing trajectory optimization. The advantage of our formulation is that it readily generalizes to a variety of scenes that might have diverse ground properties and supports any form of self-contact and contact between the articulated body and scene geometry. We show that our approach achieves competitive results with respect to existing physics-based methods on the Human3.6M benchmark, while being directly applicable without re-training to more complex dynamic motions from the AIST benchmark and to uncontrolled internet videos.