VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds

TL;DR

VoteHMR is a novel occlusion-aware voting network that accurately reconstructs 3D human meshes from single partial point clouds, effectively handling occlusions and noise for applications in AR/VR and behavior analysis.

Contribution

It introduces the first occlusion-aware voting network for single-frame partial point cloud 3D human mesh recovery, improving robustness and accuracy over prior methods.

Findings

Achieves state-of-the-art results on SURREAL and DFAUST datasets.

Demonstrates strong generalization to real-world datasets like Berkeley MHAD.

Effectively handles occlusions and noise in partial point clouds.

Abstract

3D human mesh recovery from point clouds is essential for various tasks, including AR/VR and human behavior understanding. Previous works in this field either require high-quality 3D human scans or sequential point clouds, which cannot be easily applied to low-quality 3D scans captured by consumer-level depth sensors. In this paper, we make the first attempt to reconstruct reliable 3D human shapes from single-frame partial point clouds.To achieve this, we propose an end-to-end learnable method, named VoteHMR. The core of VoteHMR is a novel occlusion-aware voting network that can first reliably produce visible joint-level features from the input partial point clouds, and then complete the joint-level features through the kinematic tree of the human skeleton. Compared with holistic features used by previous works, the joint-level features can not only effectively encode the human geometry information but also be robust to noisy inputs with self-occlusions and missing areas. By exploiting the rich complementary clues from the joint-level features and global features from the input point clouds, the proposed method encourages reliable and disentangled parameter predictions for statistical 3D human models, such as SMPL. The proposed method achieves state-of-the-art performances on two large-scale datasets, namely SURREAL and DFAUST. Furthermore, VoteHMR also demonstrates superior generalization ability on real-world datasets, such as Berkeley MHAD.