HuMoR: 3D Human Motion Model for Robust Pose Estimation

TL;DR

HuMoR is a novel 3D human motion model that uses a conditional variational autoencoder to improve pose estimation robustness against noise and occlusions, enabling accurate reconstruction from various input modalities.

Contribution

The paper introduces HuMoR, an expressive generative model for 3D human motion that acts as a prior to enhance pose estimation under challenging conditions.

Findings

HuMoR generalizes well to diverse motions and body shapes.

It enables motion reconstruction from 3D keypoints and RGB(-D) videos.

The model improves robustness in pose estimation with noisy and occluded data.

Abstract

We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from dynamic observations, recovering plausible pose sequences in the presence of noise and occlusions remains a challenge. For this purpose, we propose an expressive generative model in the form of a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. Furthermore, we introduce a flexible optimization-based approach that leverages HuMoR as a motion prior to robustly estimate plausible pose and shape from ambiguous observations. Through extensive evaluations, we demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset, and enables motion reconstruction from multiple input modalities including 3D keypoints and RGB(-D) videos.