Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose

TL;DR

Pose2Mesh introduces a graph convolutional network that directly estimates 3D human mesh vertices from 2D pose data, effectively addressing appearance domain gaps and rotation representation challenges in 3D human pose estimation.

Contribution

The paper presents a novel GraphCNN-based system that estimates 3D human mesh vertices directly from 2D poses, avoiding common issues in prior methods.

Findings

Outperforms previous methods on benchmark datasets

Effectively handles domain gap between controlled and in-the-wild images

Avoids rotation representation issues in 3D pose estimation

Abstract

Most of the recent deep learning-based 3D human pose and mesh estimation methods regress the pose and shape parameters of human mesh models, such as SMPL and MANO, from an input image. The first weakness of these methods is an appearance domain gap problem, due to different image appearance between train data from controlled environments, such as a laboratory, and test data from in-the-wild environments. The second weakness is that the estimation of the pose parameters is quite challenging owing to the representation issues of 3D rotations. To overcome the above weaknesses, we propose Pose2Mesh, a novel graph convolutional neural network (GraphCNN)-based system that estimates the 3D coordinates of human mesh vertices directly from the 2D human pose. The 2D human pose as input provides essential human body articulation information, while having a relatively homogeneous geometric property between the two domains. Also, the proposed system avoids the representation issues, while fully exploiting the mesh topology using a GraphCNN in a coarse-to-fine manner. We show that our Pose2Mesh outperforms the previous 3D human pose and mesh estimation methods on various benchmark datasets. For the codes, see https://github.com/hongsukchoi/Pose2Mesh_RELEASE.