Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Dataset

TL;DR

Point2Pose introduces a generative framework utilizing multi-view point clouds and sequential data to improve 3D human pose estimation, addressing challenges like occlusion and complex geometry.

Contribution

The paper presents a novel generative model and a large-scale multi-modal dataset for enhanced 3D human pose estimation from point cloud data.

Findings

Outperforms baseline models in pose estimation accuracy

Effective modeling of pose distribution conditioned on sequential data

Demonstrates robustness across various datasets

Abstract

We propose a novel generative approach for 3D human pose estimation. 3D human pose estimation poses several key challenges due to the complex geometry of the human body, self-occluding joints, and the requirement for large-scale real-world motion datasets. To address these challenges, we introduce Point2Pose, a framework that effectively models the distribution of human poses conditioned on sequential point cloud and pose history. Specifically, we employ a spatio-temporal point cloud encoder and a pose feature encoder to extract joint-wise features, followed by an attention-based generative regressor. Additionally, we present a large-scale indoor dataset MVPose3D, which contains multiple modalities, including IMU data of non-trivial human motions, dense multi-view point clouds, and RGB images. Experimental results show that the proposed method outperforms the baseline models, demonstrating its superior performance across various datasets.