MoPO: Incorporating Motion Prior for Occluded Human Mesh Recovery

TL;DR

MoPO leverages motion priors and a novel occlusion detection and completion approach to improve robustness and accuracy in occluded human mesh recovery, outperforming existing methods.

Contribution

The paper introduces MoPO, a method combining motion prior-based occlusion completion with fusion and refinement for better occluded human mesh recovery.

Findings

MoPO achieves state-of-the-art results on occlusion-specific benchmarks.

It significantly improves accuracy and temporal consistency in occluded scenarios.

The approach effectively estimates plausible joint positions for occluded parts.

Abstract

Although recent studies have made remarkable progress in human mesh recovery, they still exhibit limited robustness to occlusions and often produce inaccurate poses and severe motion jitter due to the insufficient spatial features for occluded body parts. Inspired by the rapid advancements in human motion prediction, we discover that compared to occluded image features, pose sequence inherently contains reliable motion prior for estimating occluded body parts. In this paper, we incorporate Motion Prior for Occluded human mesh recovery, called MoPO. Our MoPO mainly consists of two components: 1) The motion de-occlusion module, where we propose a spatial-temporal occlusion detector to detect joint visibility, and then we propose a lightweight motion predictor to complete the occluded body parts by predicting the most plausible joint positions based on history poses. 2) The motion-aware fusion and refinement module, which fuses the completed joint sequence with image features to estimate human shape and initial human pose. Moreover, the completed joint sequence is further used to refine the final human pose through inverse kinematics, which provides the occlusion-free motion prior for regressing human poses. Extensive experiments demonstrate that MoPO achieves state-of-the-art performance on both occlusion-specific and standard benchmarks, significantly enhancing the accuracy and temporal consistency of occluded human mesh recovery. Our code and demo can be found in the supplementary material.