Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture

TL;DR

This paper introduces Neural MoCon, a physics-based motion capture method that utilizes a non-differentiable physics simulator and real physical supervision to generate physically plausible human motions, especially with complex terrain interactions.

Contribution

It proposes a novel approach combining a physics simulator, an interaction constraint, and a two-branch decoder to improve the physical plausibility of human motion capture.

Findings

Achieves realistic human motion with terrain interactions

Handles diverse human shapes and behaviors

Outperforms previous kinematic methods in physical plausibility

Abstract

Due to the visual ambiguity, purely kinematic formulations on monocular human motion capture are often physically incorrect, biomechanically implausible, and can not reconstruct accurate interactions. In this work, we focus on exploiting the high-precision and non-differentiable physics simulator to incorporate dynamical constraints in motion capture. Our key-idea is to use real physical supervisions to train a target pose distribution prior for sampling-based motion control to capture physically plausible human motion. To obtain accurate reference motion with terrain interactions for the sampling, we first introduce an interaction constraint based on SDF (Signed Distance Field) to enforce appropriate ground contact modeling. We then design a novel two-branch decoder to avoid stochastic error from pseudo ground-truth and train a distribution prior with the non-differentiable physics simulator. Finally, we regress the sampling distribution from the current state of the physical character with the trained prior and sample satisfied target poses to track the estimated reference motion. Qualitative and quantitative results show that we can obtain physically plausible human motion with complex terrain interactions, human shape variations, and diverse behaviors. More information can be found at~\url{https://www.yangangwang.com/papers/HBZ-NM-2022-03.html}