Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

TL;DR

This paper introduces a probabilistic 3D human mesh recovery method that leverages uncertainty from 2D pose detectors, improving accuracy and diversity of hypotheses by combining likelihood maximization with distribution supervision and segmentation masks.

Contribution

It proposes a novel approach that combines probabilistic modeling with supervision from 2D pose heatmaps and segmentation masks to enhance 3D human mesh estimation.

Findings

Outperforms state-of-the-art probabilistic methods on 3DPW and EMDB datasets.

Utilizes segmentation masks to reduce invalid hypotheses.

Demonstrates the effectiveness of combining distribution supervision with uncertainty modeling.

Abstract

Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.